[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:50:59 dist-test-slave-c51v krb5kdc[16154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:50:59 dist-test-slave-c51v krb5kdc[16154](info): set up 2 sockets
Sep 29 15:50:59 dist-test-slave-c51v krb5kdc[16154](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260929 15:51:00.274158 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.003s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:51:00 dist-test-slave-c51v krb5kdc[16154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:51:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:51:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:51:00.445128 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:40831
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:33369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:40831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:00.766029 16167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:00.766387 16167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:00.766554 16167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:00.773555 16167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:00.773691 16167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:00.773794 16167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:00.773888 16167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:00.773967 16167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:51:00.783546 16167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:33369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:40831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:40831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:00.784838 16167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:00.786162 16167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:00.798169 16175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:00.798128 16172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:00.798208 16173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:00.804934 16167 server_base.cc:1034] running on GCE node
I20260929 15:51:00.805745 16167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:00.807199 16167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:00.808429 16167 hybrid_clock.cc:648] HybridClock initialized: now 1790697060808355 us; error 65 us; skew 500 ppm
I20260929 15:51:00.808782 16167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:00 dist-test-slave-c51v krb5kdc[16154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697060, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:00.811933 16167 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:51:00.813458 16167 webserver.cc:460] Webserver started at http://127.15.196.254:41591/ using document root <none> and password file <none>
I20260929 15:51:00.814182 16167 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:00.814325 16167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:00.814618 16167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:00.817673 16167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "03ffbd2d0c584de49a044deeb510af0c"
format_stamp: "Formatted at 2026-09-29 15:51:00 on dist-test-slave-c51v"
server_key: "22bce2434345a2bf3c79ab3e6fb63a73"
server_key_iv: "e01de33132b5ff3316dd597ebe7f9fb7"
server_key_version: "encryptionkey@0"
I20260929 15:51:00.818428 16167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "03ffbd2d0c584de49a044deeb510af0c"
format_stamp: "Formatted at 2026-09-29 15:51:00 on dist-test-slave-c51v"
server_key: "22bce2434345a2bf3c79ab3e6fb63a73"
server_key_iv: "e01de33132b5ff3316dd597ebe7f9fb7"
server_key_version: "encryptionkey@0"
I20260929 15:51:00.823663 16167 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260929 15:51:00.826937 16182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:00.828186 16167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:00.828573 16167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "03ffbd2d0c584de49a044deeb510af0c"
format_stamp: "Formatted at 2026-09-29 15:51:00 on dist-test-slave-c51v"
server_key: "22bce2434345a2bf3c79ab3e6fb63a73"
server_key_iv: "e01de33132b5ff3316dd597ebe7f9fb7"
server_key_version: "encryptionkey@0"
I20260929 15:51:00.828811 16167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:00.860638 16167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:00.864944 16167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:00.865254 16167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:00.883589 16167 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:40831
I20260929 15:51:00.883610 16233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:40831 every 8 connection(s)
I20260929 15:51:00.884991 16167 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:51:00.888641 16234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:00.891086 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 16167
I20260929 15:51:00.891399 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:51:00.893211 16147 external_mini_cluster.cc:1428] Setting key 0896c869696f889516538114459c1059
I20260929 15:51:00.900147 16234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c: Bootstrap starting.
I20260929 15:51:00.902976 16234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:00.904016 16234 log.cc:824] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:00.906632 16234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c: No bootstrap required, opened a new log
I20260929 15:51:00.913908 16234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ffbd2d0c584de49a044deeb510af0c" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40831 } }
I20260929 15:51:00.914279 16234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:00.914415 16234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03ffbd2d0c584de49a044deeb510af0c, State: Initialized, Role: FOLLOWER
I20260929 15:51:00.914973 16234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ffbd2d0c584de49a044deeb510af0c" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40831 } }
I20260929 15:51:00.915249 16234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:51:00.915429 16234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:51:00.915606 16234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:00.918054 16234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ffbd2d0c584de49a044deeb510af0c" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40831 } }
I20260929 15:51:00.918526 16234 leader_election.cc:302] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03ffbd2d0c584de49a044deeb510af0c; no voters:
I20260929 15:51:00.919121 16234 leader_election.cc:288] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:51:00.919276 16239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [term 1 FOLLOWER]: Leader election won for term 1
Sep 29 15:51:00 dist-test-slave-c51v krb5kdc[16154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:00.921203 16234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:00.923725 16239 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [term 1 LEADER]: Becoming Leader. State: Replica: 03ffbd2d0c584de49a044deeb510af0c, State: Running, Role: LEADER
I20260929 15:51:00.924367 16239 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ffbd2d0c584de49a044deeb510af0c" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40831 } }
I20260929 15:51:00.927748 16241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03ffbd2d0c584de49a044deeb510af0c. Latest consensus state: current_term: 1 leader_uuid: "03ffbd2d0c584de49a044deeb510af0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ffbd2d0c584de49a044deeb510af0c" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40831 } } }
I20260929 15:51:00.927801 16240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03ffbd2d0c584de49a044deeb510af0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ffbd2d0c584de49a044deeb510af0c" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40831 } } }
I20260929 15:51:00.928201 16241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:00.928207 16240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:00.929791 16247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:00.937011 16236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:00.896976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45732 (local address 127.15.196.254:40831)
0929 15:51:00.897933 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:00.897950 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:00.898964 (+ 1014us) server_negotiation.cc:407] Connection header received
0929 15:51:00.907734 (+ 8770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:00.907760 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:00.908119 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:00.908519 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:00.911280 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:00.912677 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:00.914287 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:00.915142 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:00.915365 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:00.922646 (+ 7281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:00.922668 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:00.922696 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:00.922748 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:00.928273 (+ 5525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:00.929307 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:00.929318 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:00.929329 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:00.929684 (+ 355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:00.930393 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:00.930398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:00.930401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:00.931191 (+ 790us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:00.932207 (+ 1016us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:00.934823 (+ 2616us) server_negotiation.cc:299] Negotiation successful
0929 15:51:00.935410 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":452,"threads_started":1}
I20260929 15:51:00.936157 16247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:00.952654 16247 catalog_manager.cc:1347] Generated new cluster ID: 7353dfb9ad19408f81070d1dac983b73
I20260929 15:51:00.952785 16247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:00.962157 16247 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:51:00.964110 16247 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:00.974159 16247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03ffbd2d0c584de49a044deeb510af0c: Generated new TSK 0
I20260929 15:51:00.974983 16247 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:51:01.024390 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:40831
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:33369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:01.366495 16260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:01.366777 16260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:01.366938 16260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:01.374598 16260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:01.374713 16260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:01.374989 16260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:51:01.386122 16260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:33369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:40831
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:01.387590 16260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:01.389396 16260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:01.403352 16265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:01.403386 16266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:01.403563 16268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:01.404286 16260 server_base.cc:1034] running on GCE node
I20260929 15:51:01.404922 16260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:01.405951 16260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:01.407214 16260 hybrid_clock.cc:648] HybridClock initialized: now 1790697061407162 us; error 39 us; skew 500 ppm
I20260929 15:51:01.407593 16260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:01 dist-test-slave-c51v krb5kdc[16154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:01.410705 16260 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:51:01.412238 16260 webserver.cc:460] Webserver started at http://127.15.196.193:44559/ using document root <none> and password file <none>
I20260929 15:51:01.413172 16260 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:01.413353 16260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:01.413763 16260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:01.416980 16260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "33ff5b1db9b845e688951b90545350ea"
format_stamp: "Formatted at 2026-09-29 15:51:01 on dist-test-slave-c51v"
server_key: "115344efb4af44606fbd9102b27a9fef"
server_key_iv: "7f5791c82b0a20e903af6fb7b557c4ce"
server_key_version: "encryptionkey@0"
I20260929 15:51:01.417843 16260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "33ff5b1db9b845e688951b90545350ea"
format_stamp: "Formatted at 2026-09-29 15:51:01 on dist-test-slave-c51v"
server_key: "115344efb4af44606fbd9102b27a9fef"
server_key_iv: "7f5791c82b0a20e903af6fb7b557c4ce"
server_key_version: "encryptionkey@0"
I20260929 15:51:01.424258 16260 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260929 15:51:01.428071 16275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:01.429567 16260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:51:01.429762 16260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "33ff5b1db9b845e688951b90545350ea"
format_stamp: "Formatted at 2026-09-29 15:51:01 on dist-test-slave-c51v"
server_key: "115344efb4af44606fbd9102b27a9fef"
server_key_iv: "7f5791c82b0a20e903af6fb7b557c4ce"
server_key_version: "encryptionkey@0"
I20260929 15:51:01.429963 16260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:01.448383 16260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:01.452353 16260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:01.452644 16260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:01.453630 16260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:01.454965 16260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:01.455068 16260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:01.455215 16260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:01.455286 16260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:01.489540 16260 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:34835
I20260929 15:51:01.489575 16387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:34835 every 8 connection(s)
I20260929 15:51:01.490882 16260 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:51:01.494338 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 16260
I20260929 15:51:01.494681 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:51:01.495446 16147 external_mini_cluster.cc:1428] Setting key 3b796ec59e856e4a4597bb289850b5c5
Sep 29 15:51:01 dist-test-slave-c51v krb5kdc[16154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
I20260929 15:51:01.512156 16390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:01.493494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:45135 (local address 127.15.196.254:40831)
0929 15:51:01.493986 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:01.493992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:01.495124 (+ 1132us) server_negotiation.cc:407] Connection header received
0929 15:51:01.497179 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:01.497183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:01.497284 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:01.497421 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:01.498876 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:01.500356 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:01.502332 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:01.502721 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:01.502834 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:01.506365 (+ 3531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:01.506370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:01.506374 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:01.506405 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:01.509138 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:01.509866 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:01.509874 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:01.509878 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:01.510001 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:01.510424 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:01.510429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:01.510432 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:01.510688 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:01.510847 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:01.511557 (+ 710us) server_negotiation.cc:299] Negotiation successful
0929 15:51:01.511770 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":224,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
I20260929 15:51:01.513283 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:01.494026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:40831 (local address 127.15.196.193:45135)
0929 15:51:01.494945 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:01.494994 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:01.496892 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:01.497620 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:01.497631 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:01.498149 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:01.498697 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:01.498714 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:01.500528 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:01.500532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:01.502162 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:01.502181 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:01.502980 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:01.502986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:01.503228 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:01.503970 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:01.503997 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:01.506149 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:01.509283 (+ 3134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:01.509292 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:01.509308 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:01.509745 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:01.510119 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:01.510123 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:01.510127 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:01.510305 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:01.510817 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:01.510825 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:01.511158 (+ 333us) client_negotiation.cc:769] Sending connection context
0929 15:51:01.511475 (+ 317us) client_negotiation.cc:240] Negotiation successful
0929 15:51:01.512037 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":327,"threads_started":1}
I20260929 15:51:01.514816 16388 heartbeater.cc:347] Connected to a master server at 127.15.196.254:40831
I20260929 15:51:01.515105 16388 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:01.515923 16388 heartbeater.cc:511] Master 127.15.196.254:40831 requested a full tablet report, sending...
I20260929 15:51:01.518157 16199 ts_manager.cc:194] Registered new tserver with Master: 33ff5b1db9b845e688951b90545350ea (127.15.196.193:34835)
I20260929 15:51:01.519755 16199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:45135
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:51:01.532825 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:40831
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:33369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:01.885910 16394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:01.886193 16394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:01.886372 16394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:01.893815 16394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:01.893930 16394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:01.894141 16394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:51:01.904233 16394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:33369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:40831
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:01.905546 16394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:01.906985 16394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:01.920485 16399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:01.920625 16402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:01.920660 16400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:01.921383 16394 server_base.cc:1034] running on GCE node
I20260929 15:51:01.922029 16394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:01.922994 16394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:01.924185 16394 hybrid_clock.cc:648] HybridClock initialized: now 1790697061924132 us; error 43 us; skew 500 ppm
I20260929 15:51:01.924508 16394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:01 dist-test-slave-c51v krb5kdc[16154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:01.927788 16394 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:51:01.929168 16394 webserver.cc:460] Webserver started at http://127.15.196.194:41887/ using document root <none> and password file <none>
I20260929 15:51:01.929849 16394 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:01.929994 16394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:01.930303 16394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:01.933483 16394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "e557ada0a0834b54bc643cf87aeebae3"
format_stamp: "Formatted at 2026-09-29 15:51:01 on dist-test-slave-c51v"
server_key: "37ca86d5dfad42408d27a37ff087e72f"
server_key_iv: "553d08ee79aa4400d6587694f9b1abcc"
server_key_version: "encryptionkey@0"
I20260929 15:51:01.934273 16394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "e557ada0a0834b54bc643cf87aeebae3"
format_stamp: "Formatted at 2026-09-29 15:51:01 on dist-test-slave-c51v"
server_key: "37ca86d5dfad42408d27a37ff087e72f"
server_key_iv: "553d08ee79aa4400d6587694f9b1abcc"
server_key_version: "encryptionkey@0"
I20260929 15:51:01.939457 16394 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 15:51:01.942710 16409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:01.944056 16394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:01.944219 16394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "e557ada0a0834b54bc643cf87aeebae3"
format_stamp: "Formatted at 2026-09-29 15:51:01 on dist-test-slave-c51v"
server_key: "37ca86d5dfad42408d27a37ff087e72f"
server_key_iv: "553d08ee79aa4400d6587694f9b1abcc"
server_key_version: "encryptionkey@0"
I20260929 15:51:01.944422 16394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:01.961732 16394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:01.966089 16394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:01.966387 16394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:01.967379 16394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:01.968725 16394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:01.968842 16394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:01.968992 16394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:01.969063 16394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:02.003950 16394 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:46819
I20260929 15:51:02.003996 16521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:46819 every 8 connection(s)
I20260929 15:51:02.005300 16394 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:51:02.009544 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 16394
I20260929 15:51:02.009749 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:51:02.010172 16147 external_mini_cluster.cc:1428] Setting key 1de0acfff587686aa70d8955daadcd05
Sep 29 15:51:02 dist-test-slave-c51v krb5kdc[16154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697061, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
I20260929 15:51:02.027496 16390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:02.007916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:42789 (local address 127.15.196.254:40831)
0929 15:51:02.008087 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:02.008092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:02.009342 (+ 1250us) server_negotiation.cc:407] Connection header received
0929 15:51:02.011450 (+ 2108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:02.011454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:02.011523 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:02.011611 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:02.013394 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.014593 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.016748 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.017171 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.017316 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:02.020765 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:02.020771 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:02.020776 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:02.020808 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:02.024401 (+ 3593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:02.025127 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:02.025133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:02.025136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:02.025309 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:02.025718 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:02.025724 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:02.025727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:02.026141 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:02.026288 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:02.026990 (+ 702us) server_negotiation.cc:299] Negotiation successful
0929 15:51:02.027161 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 15:51:02.028616 16524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:02.008463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:40831 (local address 127.15.196.194:42789)
0929 15:51:02.009180 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:02.009224 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:02.011117 (+ 1893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:02.011891 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:02.011902 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:02.012590 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:02.013214 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:02.013232 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.014757 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.014764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:02.016603 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:02.016615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.017416 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.017424 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:02.017653 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:02.018362 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:02.018388 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:02.020539 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:02.024541 (+ 4002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:02.024549 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:02.024563 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:02.024987 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:02.025417 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:02.025420 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:02.025423 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:02.025595 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:02.026257 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:02.026263 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:02.026634 (+ 371us) client_negotiation.cc:769] Sending connection context
0929 15:51:02.026878 (+ 244us) client_negotiation.cc:240] Negotiation successful
0929 15:51:02.027260 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":270,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
I20260929 15:51:02.030161 16522 heartbeater.cc:347] Connected to a master server at 127.15.196.254:40831
I20260929 15:51:02.030530 16522 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:02.031508 16522 heartbeater.cc:511] Master 127.15.196.254:40831 requested a full tablet report, sending...
I20260929 15:51:02.033293 16199 ts_manager.cc:194] Registered new tserver with Master: e557ada0a0834b54bc643cf87aeebae3 (127.15.196.194:46819)
I20260929 15:51:02.034255 16199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:42789
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:51:02.052001 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:40831
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:33369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:02.374954 16527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:02.375277 16527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:02.375494 16527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:02.382885 16527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:02.383034 16527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:02.383219 16527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:51:02.393108 16527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:33369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:40831
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:02.394440 16527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:02.395870 16527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:02.409230 16532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:02.409201 16533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:02.409195 16535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:02.410256 16527 server_base.cc:1034] running on GCE node
I20260929 15:51:02.410897 16527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:02.411924 16527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:02.413162 16527 hybrid_clock.cc:648] HybridClock initialized: now 1790697062413110 us; error 45 us; skew 500 ppm
I20260929 15:51:02.413494 16527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:02 dist-test-slave-c51v krb5kdc[16154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697062, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:02.416802 16527 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:51:02.418329 16527 webserver.cc:460] Webserver started at http://127.15.196.195:38843/ using document root <none> and password file <none>
I20260929 15:51:02.419034 16527 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:02.419195 16527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:02.419560 16527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:02.422631 16527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "29f84ef729d54ef197cbbf2ae7b041f8"
format_stamp: "Formatted at 2026-09-29 15:51:02 on dist-test-slave-c51v"
server_key: "1d926b3ceb0ec3bb44792b0906c0bc40"
server_key_iv: "10c951ee6cfd2c1774363a95371a494f"
server_key_version: "encryptionkey@0"
I20260929 15:51:02.423511 16527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "29f84ef729d54ef197cbbf2ae7b041f8"
format_stamp: "Formatted at 2026-09-29 15:51:02 on dist-test-slave-c51v"
server_key: "1d926b3ceb0ec3bb44792b0906c0bc40"
server_key_iv: "10c951ee6cfd2c1774363a95371a494f"
server_key_version: "encryptionkey@0"
I20260929 15:51:02.430065 16527 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260929 15:51:02.435066 16542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:02.436491 16527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:02.436679 16527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "29f84ef729d54ef197cbbf2ae7b041f8"
format_stamp: "Formatted at 2026-09-29 15:51:02 on dist-test-slave-c51v"
server_key: "1d926b3ceb0ec3bb44792b0906c0bc40"
server_key_iv: "10c951ee6cfd2c1774363a95371a494f"
server_key_version: "encryptionkey@0"
I20260929 15:51:02.436905 16527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:02.457644 16527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:02.462001 16527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:02.462291 16527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:02.463230 16527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:02.464525 16527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:02.464632 16527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:02.464776 16527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:02.464826 16527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:02.497970 16527 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:38133
I20260929 15:51:02.498019 16654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:38133 every 8 connection(s)
I20260929 15:51:02.499258 16527 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:51:02.508679 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 16527
I20260929 15:51:02.508904 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:51:02.509465 16147 external_mini_cluster.cc:1428] Setting key 37b84116c124e9916e5301232cea966a
Sep 29 15:51:02 dist-test-slave-c51v krb5kdc[16154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697062, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:02.519613 16390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:02.501832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:58005 (local address 127.15.196.254:40831)
0929 15:51:02.502036 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:02.502042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:02.503266 (+ 1224us) server_negotiation.cc:407] Connection header received
0929 15:51:02.505081 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:02.505085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:02.505188 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:02.505287 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:02.506740 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.507526 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.509700 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.510344 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.510448 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:02.514010 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:02.514019 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:02.514028 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:02.514060 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:02.516582 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:02.517291 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:02.517298 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:02.517303 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:02.517456 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:02.517882 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:02.517888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:02.517891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:02.518157 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:02.518312 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:02.519004 (+ 692us) server_negotiation.cc:299] Negotiation successful
0929 15:51:02.519221 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 15:51:02.520620 16657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:02.502367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:40831 (local address 127.15.196.195:58005)
0929 15:51:02.503100 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:02.503138 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:02.504821 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:02.505491 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:02.505499 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:02.506044 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:02.506591 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:02.506607 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.507659 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.507663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:02.509485 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:02.509496 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.510569 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.510576 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:02.510807 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:02.511549 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:02.511570 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:02.513817 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:02.516717 (+ 2900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:02.516727 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:02.516744 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:02.517159 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:02.517569 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:02.517574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:02.517578 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:02.517770 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:02.518290 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:02.518298 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:02.518630 (+ 332us) client_negotiation.cc:769] Sending connection context
0929 15:51:02.518918 (+ 288us) client_negotiation.cc:240] Negotiation successful
0929 15:51:02.519461 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":300,"threads_started":1}
I20260929 15:51:02.522025 16655 heartbeater.cc:347] Connected to a master server at 127.15.196.254:40831
I20260929 15:51:02.522358 16655 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:02.522653 16388 heartbeater.cc:503] Master 127.15.196.254:40831 was elected leader, sending a full tablet report...
I20260929 15:51:02.523247 16655 heartbeater.cc:511] Master 127.15.196.254:40831 requested a full tablet report, sending...
I20260929 15:51:02.524853 16198 ts_manager.cc:194] Registered new tserver with Master: 29f84ef729d54ef197cbbf2ae7b041f8 (127.15.196.195:38133)
I20260929 15:51:02.525750 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:51:02.525880 16198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:58005
Sep 29 15:51:02 dist-test-slave-c51v krb5kdc[16154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697062, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 15:51:02 dist-test-slave-c51v krb5kdc[16154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697062, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:51:02.571724 16662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:02.556428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44938 (local address 127.15.196.193:34835)
0929 15:51:02.556983 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:02.556991 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:02.557501 (+ 510us) server_negotiation.cc:407] Connection header received
0929 15:51:02.557637 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:02.557644 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:02.557877 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:02.558009 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:02.559479 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.560471 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.561666 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.562224 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.562330 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:02.566271 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:02.566286 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:02.566299 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:02.566337 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:02.569324 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:02.569929 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:02.569935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:02.569937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:02.570051 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:02.570401 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:02.570404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:02.570406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:02.570715 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:02.570953 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:02.571168 (+ 215us) server_negotiation.cc:299] Negotiation successful
0929 15:51:02.571450 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":256,"threads_started":1}
W20260929 15:51:02.572952 16303 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44938
Sep 29 15:51:02 dist-test-slave-c51v krb5kdc[16154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 15:51:02 dist-test-slave-c51v krb5kdc[16154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:51:02.614537 16662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:02.601159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44954 (local address 127.15.196.193:34835)
0929 15:51:02.601400 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:02.601407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:02.601634 (+ 227us) server_negotiation.cc:407] Connection header received
0929 15:51:02.601873 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:02.601877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:02.601969 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:02.602079 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:02.603469 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.604155 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.605358 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.605762 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.605900 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:02.609648 (+ 3748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:02.609653 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:02.609656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:02.609701 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:02.612156 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:02.612712 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:02.612716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:02.612718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:02.612842 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:02.613237 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:02.613242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:02.613244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:02.613548 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:02.613788 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:02.614079 (+ 291us) server_negotiation.cc:299] Negotiation successful
0929 15:51:02.614275 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260929 15:51:02.616214 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 16260
I20260929 15:51:02.628695 16662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:02.617233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44966 (local address 127.15.196.193:34835)
0929 15:51:02.617451 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:02.617458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:02.617657 (+ 199us) server_negotiation.cc:407] Connection header received
0929 15:51:02.617850 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:02.617854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:02.617906 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:02.618004 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:02.619153 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.619900 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.621127 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.621622 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.621760 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:02.624226 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:02.624231 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:02.624234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:02.624261 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:02.626597 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:02.627163 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:02.627166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:02.627169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:02.627351 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:02.627656 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:02.627659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:02.627661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:02.627889 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:02.628049 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:02.628297 (+ 248us) server_negotiation.cc:299] Negotiation successful
0929 15:51:02.628442 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 15:51:02.629453 16383 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:02.713538 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 16260
I20260929 15:51:02.730527 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 16394
Sep 29 15:51:02 dist-test-slave-c51v krb5kdc[16154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:51:02.745956 16669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:02.731422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56748 (local address 127.15.196.194:46819)
0929 15:51:02.732005 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:02.732013 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:02.732039 (+ 26us) server_negotiation.cc:407] Connection header received
0929 15:51:02.732150 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:02.732156 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:02.732388 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:02.732520 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:02.733676 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.734623 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.735941 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.736471 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.736568 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:02.740625 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:02.740637 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:02.740653 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:02.740686 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:02.743590 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:02.744202 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:02.744209 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:02.744211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:02.744326 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:02.744635 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:02.744638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:02.744640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:02.745004 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:02.745226 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:02.745453 (+ 227us) server_negotiation.cc:299] Negotiation successful
0929 15:51:02.745638 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":274,"threads_started":1}
I20260929 15:51:02.747262 16517 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:02.828819 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 16394
I20260929 15:51:02.851331 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 16527
Sep 29 15:51:02 dist-test-slave-c51v krb5kdc[16154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:51:02.867790 16671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:02.852273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40980 (local address 127.15.196.195:38133)
0929 15:51:02.852862 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:02.852869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:02.852897 (+ 28us) server_negotiation.cc:407] Connection header received
0929 15:51:02.852956 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:02.852962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:02.853187 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:02.853372 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:02.854680 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.855908 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.857211 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:02.857798 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:02.857935 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:02.862115 (+ 4180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:02.862126 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:02.862142 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:02.862182 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:02.864930 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:02.865617 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:02.865624 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:02.865626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:02.865753 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:02.866120 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:02.866127 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:02.866131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:02.866488 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:02.866798 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:02.867129 (+ 331us) server_negotiation.cc:299] Negotiation successful
0929 15:51:02.867447 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":299,"threads_started":1}
I20260929 15:51:02.869062 16650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:02.955127 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 16527
I20260929 15:51:02.971483 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 16167
I20260929 15:51:02.972692 16229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:03.077972 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 16167
2026-09-29T15:51:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3911 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:51:03 dist-test-slave-c51v krb5kdc[16676](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:51:03 dist-test-slave-c51v krb5kdc[16676](info): set up 2 sockets
Sep 29 15:51:03 dist-test-slave-c51v krb5kdc[16676](info): commencing operation
krb5kdc: starting...
W20260929 15:51:04.149258 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.004s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:51:04 dist-test-slave-c51v krb5kdc[16676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:51:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:51:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:51:04.322672 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:44427
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:44427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:04.636152 16689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:04.636514 16689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:04.636684 16689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:04.643777 16689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:04.643920 16689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:04.644047 16689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:04.644155 16689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:04.644222 16689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:51:04.653795 16689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:44427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:44427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:04.655139 16689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:04.656486 16689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:04.668298 16694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:04.668267 16695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:04.668231 16697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:04.669092 16689 server_base.cc:1034] running on GCE node
I20260929 15:51:04.669952 16689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:04.671437 16689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:04.672667 16689 hybrid_clock.cc:648] HybridClock initialized: now 1790697064672614 us; error 61 us; skew 500 ppm
I20260929 15:51:04.673032 16689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:04 dist-test-slave-c51v krb5kdc[16676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697064, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:04.679562 16689 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:51:04.681171 16689 webserver.cc:460] Webserver started at http://127.15.196.254:37609/ using document root <none> and password file <none>
I20260929 15:51:04.681923 16689 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:04.682075 16689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:04.682408 16689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:04.685627 16689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "cd415f2f8f8646c7a84872be1c2d9961"
format_stamp: "Formatted at 2026-09-29 15:51:04 on dist-test-slave-c51v"
server_key: "91b93de591f29459890781aff751b217"
server_key_iv: "47350376b3be7da08a2fb068a6ba272c"
server_key_version: "encryptionkey@0"
I20260929 15:51:04.686451 16689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "cd415f2f8f8646c7a84872be1c2d9961"
format_stamp: "Formatted at 2026-09-29 15:51:04 on dist-test-slave-c51v"
server_key: "91b93de591f29459890781aff751b217"
server_key_iv: "47350376b3be7da08a2fb068a6ba272c"
server_key_version: "encryptionkey@0"
I20260929 15:51:04.691811 16689 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260929 15:51:04.695253 16704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:04.696664 16689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:51:04.696871 16689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "cd415f2f8f8646c7a84872be1c2d9961"
format_stamp: "Formatted at 2026-09-29 15:51:04 on dist-test-slave-c51v"
server_key: "91b93de591f29459890781aff751b217"
server_key_iv: "47350376b3be7da08a2fb068a6ba272c"
server_key_version: "encryptionkey@0"
I20260929 15:51:04.697072 16689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:04.721148 16689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:04.725138 16689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:04.725423 16689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:04.743834 16689 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:44427
I20260929 15:51:04.743841 16755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:44427 every 8 connection(s)
I20260929 15:51:04.745245 16689 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:51:04.749109 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 16689
I20260929 15:51:04.749182 16756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:04.749363 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:51:04.750154 16147 external_mini_cluster.cc:1428] Setting key bb9317cfbbd8be73a32dab85dd7b983d
I20260929 15:51:04.763549 16756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961: Bootstrap starting.
Sep 29 15:51:04 dist-test-slave-c51v krb5kdc[16676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:04.767185 16756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:04.768489 16756 log.cc:824] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:04.771433 16756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961: No bootstrap required, opened a new log
I20260929 15:51:04.773911 16759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:04.752295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55862 (local address 127.15.196.254:44427)
0929 15:51:04.753104 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:04.753116 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:04.753157 (+ 41us) server_negotiation.cc:407] Connection header received
0929 15:51:04.755500 (+ 2343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:04.755525 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:04.755942 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:04.756280 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:04.758121 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:04.759470 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:04.761302 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:04.761971 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:04.762102 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:04.766669 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:04.766678 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:04.766693 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:04.766732 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:04.769803 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:04.770474 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:04.770481 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:04.770488 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:04.770651 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:04.771031 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:04.771036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:04.771039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:04.771656 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:04.771966 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:04.772455 (+ 489us) server_negotiation.cc:299] Negotiation successful
0929 15:51:04.772871 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":403,"threads_started":1}
I20260929 15:51:04.779387 16756 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd415f2f8f8646c7a84872be1c2d9961" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44427 } }
I20260929 15:51:04.779861 16756 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:04.779980 16756 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd415f2f8f8646c7a84872be1c2d9961, State: Initialized, Role: FOLLOWER
I20260929 15:51:04.780619 16756 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd415f2f8f8646c7a84872be1c2d9961" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44427 } }
I20260929 15:51:04.780853 16756 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:51:04.780984 16756 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:51:04.781148 16756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:04.783253 16756 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd415f2f8f8646c7a84872be1c2d9961" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44427 } }
I20260929 15:51:04.783818 16756 leader_election.cc:302] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd415f2f8f8646c7a84872be1c2d9961; no voters:
I20260929 15:51:04.784418 16756 leader_election.cc:288] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:51:04.784578 16761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:04.785663 16761 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [term 1 LEADER]: Becoming Leader. State: Replica: cd415f2f8f8646c7a84872be1c2d9961, State: Running, Role: LEADER
I20260929 15:51:04.786185 16761 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd415f2f8f8646c7a84872be1c2d9961" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44427 } }
I20260929 15:51:04.786625 16756 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:04.788933 16762 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd415f2f8f8646c7a84872be1c2d9961" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd415f2f8f8646c7a84872be1c2d9961" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44427 } } }
I20260929 15:51:04.789235 16762 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:04.790059 16769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:04.789927 16763 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd415f2f8f8646c7a84872be1c2d9961. Latest consensus state: current_term: 1 leader_uuid: "cd415f2f8f8646c7a84872be1c2d9961" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd415f2f8f8646c7a84872be1c2d9961" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44427 } } }
I20260929 15:51:04.791117 16763 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961 [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:04.795562 16769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:04.804714 16769 catalog_manager.cc:1347] Generated new cluster ID: f9049d0fd3384e31bc9dcdead78b4410
I20260929 15:51:04.804843 16769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:04.815593 16769 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:51:04.817428 16769 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:04.829276 16769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd415f2f8f8646c7a84872be1c2d9961: Generated new TSK 0
I20260929 15:51:04.830307 16769 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:51:04.879739 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44427
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:05.207942 16782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:05.208209 16782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:05.208302 16782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:05.215469 16782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:05.215579 16782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:05.215692 16782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:51:05.225397 16782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44427
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:05.226650 16782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:05.227941 16782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:05.241303 16787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:05.241344 16790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:05.241330 16788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:05.242250 16782 server_base.cc:1034] running on GCE node
I20260929 15:51:05.242892 16782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:05.243899 16782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:05.245131 16782 hybrid_clock.cc:648] HybridClock initialized: now 1790697065245087 us; error 53 us; skew 500 ppm
I20260929 15:51:05.245462 16782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:05 dist-test-slave-c51v krb5kdc[16676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697065, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:05.248471 16782 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:51:05.249926 16782 webserver.cc:460] Webserver started at http://127.15.196.193:46079/ using document root <none> and password file <none>
I20260929 15:51:05.250677 16782 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:05.250830 16782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:05.251154 16782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:05.254199 16782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "e4ef1504475c49299790078d1ed114c1"
format_stamp: "Formatted at 2026-09-29 15:51:05 on dist-test-slave-c51v"
server_key: "9f9b5575819ad28054888f0880e62da1"
server_key_iv: "634885b2c4a674c9ef25241012eb1918"
server_key_version: "encryptionkey@0"
I20260929 15:51:05.255004 16782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "e4ef1504475c49299790078d1ed114c1"
format_stamp: "Formatted at 2026-09-29 15:51:05 on dist-test-slave-c51v"
server_key: "9f9b5575819ad28054888f0880e62da1"
server_key_iv: "634885b2c4a674c9ef25241012eb1918"
server_key_version: "encryptionkey@0"
I20260929 15:51:05.260391 16782 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 15:51:05.264289 16797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:05.265859 16782 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:51:05.266057 16782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "e4ef1504475c49299790078d1ed114c1"
format_stamp: "Formatted at 2026-09-29 15:51:05 on dist-test-slave-c51v"
server_key: "9f9b5575819ad28054888f0880e62da1"
server_key_iv: "634885b2c4a674c9ef25241012eb1918"
server_key_version: "encryptionkey@0"
I20260929 15:51:05.266304 16782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:05.308349 16782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:05.312783 16782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:05.313108 16782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:05.314144 16782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:05.315651 16782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:05.315778 16782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:05.315943 16782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:05.316022 16782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:05.350865 16782 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:32881
I20260929 15:51:05.350885 16909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:32881 every 8 connection(s)
I20260929 15:51:05.352396 16782 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:51:05.357445 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 16782
I20260929 15:51:05.357688 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:51:05.358363 16147 external_mini_cluster.cc:1428] Setting key b5b17f5fabb0f8aa7ea2a522aacc078b
Sep 29 15:51:05 dist-test-slave-c51v krb5kdc[16676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697065, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:05.374045 16912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:05.355091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:53213 (local address 127.15.196.254:44427)
0929 15:51:05.355622 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:05.355628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:05.356646 (+ 1018us) server_negotiation.cc:407] Connection header received
0929 15:51:05.358586 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:05.358592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:05.358689 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:05.358817 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:05.360650 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:05.361385 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:05.362965 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:05.363438 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:05.363576 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:05.367279 (+ 3703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:05.367335 (+ 56us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:05.367342 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:05.367381 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:05.370136 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:05.371059 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:05.371068 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:05.371075 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:05.371277 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:05.371829 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:05.371836 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:05.371841 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:05.372299 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:05.372470 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:05.373265 (+ 795us) server_negotiation.cc:299] Negotiation successful
0929 15:51:05.373536 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":273,"threads_started":1}
I20260929 15:51:05.374722 16913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:05.355773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44427 (local address 127.15.196.193:53213)
0929 15:51:05.356476 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:05.356514 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:05.358285 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:05.359065 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:05.359073 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:05.359793 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:05.360482 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:05.360499 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:05.361546 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:05.361550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:05.362814 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:05.362825 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:05.363673 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:05.363681 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:05.363922 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:05.364641 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:05.364673 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:05.367055 (+ 2382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:05.370305 (+ 3250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:05.370315 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:05.370340 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:05.370792 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:05.371462 (+ 670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:05.371476 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:05.371478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:05.371697 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:05.372451 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:05.372457 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:05.372836 (+ 379us) client_negotiation.cc:769] Sending connection context
0929 15:51:05.373085 (+ 249us) client_negotiation.cc:240] Negotiation successful
0929 15:51:05.373511 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":264,"threads_started":1}
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
I20260929 15:51:05.376447 16910 heartbeater.cc:347] Connected to a master server at 127.15.196.254:44427
I20260929 15:51:05.376793 16910 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
I20260929 15:51:05.377759 16910 heartbeater.cc:511] Master 127.15.196.254:44427 requested a full tablet report, sending...
I20260929 15:51:05.380283 16721 ts_manager.cc:194] Registered new tserver with Master: e4ef1504475c49299790078d1ed114c1 (127.15.196.193:32881)
I20260929 15:51:05.382439 16721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:53213
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:51:05.400194 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44427
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:05.720137 16916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:05.720484 16916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:05.720664 16916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:05.728036 16916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:05.728185 16916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:05.728384 16916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:51:05.738444 16916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44427
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:05.739799 16916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:05.741155 16916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:05.753901 16921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:05.754004 16922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:05.754743 16924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:05.755095 16916 server_base.cc:1034] running on GCE node
I20260929 15:51:05.755800 16916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:05.756712 16916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:05.757938 16916 hybrid_clock.cc:648] HybridClock initialized: now 1790697065757888 us; error 45 us; skew 500 ppm
I20260929 15:51:05.758256 16916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:05 dist-test-slave-c51v krb5kdc[16676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697065, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:05.761413 16916 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:51:05.762912 16916 webserver.cc:460] Webserver started at http://127.15.196.194:41677/ using document root <none> and password file <none>
I20260929 15:51:05.763665 16916 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:05.763816 16916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:05.764117 16916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:05.767196 16916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "15bcc662232a4103b30ca2ae37e1384b"
format_stamp: "Formatted at 2026-09-29 15:51:05 on dist-test-slave-c51v"
server_key: "3f9c9e45f2bbe7f9c5356ffe86ce0ee7"
server_key_iv: "fed7ecba5d0f092ffe0edc8503457e92"
server_key_version: "encryptionkey@0"
I20260929 15:51:05.768136 16916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "15bcc662232a4103b30ca2ae37e1384b"
format_stamp: "Formatted at 2026-09-29 15:51:05 on dist-test-slave-c51v"
server_key: "3f9c9e45f2bbe7f9c5356ffe86ce0ee7"
server_key_iv: "fed7ecba5d0f092ffe0edc8503457e92"
server_key_version: "encryptionkey@0"
I20260929 15:51:05.773653 16916 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:51:05.777310 16931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:05.778631 16916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:51:05.778848 16916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "15bcc662232a4103b30ca2ae37e1384b"
format_stamp: "Formatted at 2026-09-29 15:51:05 on dist-test-slave-c51v"
server_key: "3f9c9e45f2bbe7f9c5356ffe86ce0ee7"
server_key_iv: "fed7ecba5d0f092ffe0edc8503457e92"
server_key_version: "encryptionkey@0"
I20260929 15:51:05.779044 16916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:05.802181 16916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:05.806492 16916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:05.806779 16916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:05.807796 16916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:05.809094 16916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:05.809195 16916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:05.809346 16916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:05.809417 16916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:05.842423 16916 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:33299
I20260929 15:51:05.842540 17043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:33299 every 8 connection(s)
I20260929 15:51:05.843863 16916 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:51:05.846941 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 16916
I20260929 15:51:05.847137 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:51:05.847765 16147 external_mini_cluster.cc:1428] Setting key 15b6b46fd891cdd3ef1f45d4ace424cd
Sep 29 15:51:05 dist-test-slave-c51v krb5kdc[16676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697065, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
I20260929 15:51:05.865872 16912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:05.846441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:46099 (local address 127.15.196.254:44427)
0929 15:51:05.846633 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:05.846638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:05.848251 (+ 1613us) server_negotiation.cc:407] Connection header received
0929 15:51:05.850630 (+ 2379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:05.850634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:05.850699 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:05.850795 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:05.853154 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:05.853863 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:05.855809 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:05.856245 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:05.856382 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:05.860033 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:05.860038 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:05.860042 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:05.860075 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:05.862677 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:05.863497 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:05.863501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:05.863504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:05.863639 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:05.864088 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:05.864092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:05.864094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:05.864370 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:05.864526 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:05.865275 (+ 749us) server_negotiation.cc:299] Negotiation successful
0929 15:51:05.865509 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
I20260929 15:51:05.867079 17046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:05.847149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44427 (local address 127.15.196.194:46099)
0929 15:51:05.848070 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:05.848117 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:05.850257 (+ 2140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:05.851090 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:05.851103 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:05.852007 (+ 904us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:05.852943 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:05.852973 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:05.854035 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:05.854044 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:05.855662 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:05.855673 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:05.856475 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:05.856482 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:05.856715 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:05.857464 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:05.857487 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:05.859780 (+ 2293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:05.862840 (+ 3060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:05.862850 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:05.862869 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:05.863356 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:05.863754 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:05.863759 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:05.863763 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:05.863963 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:05.864504 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:05.864512 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:05.864908 (+ 396us) client_negotiation.cc:769] Sending connection context
0929 15:51:05.865173 (+ 265us) client_negotiation.cc:240] Negotiation successful
0929 15:51:05.865812 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":362,"threads_started":1}
I20260929 15:51:05.868662 17044 heartbeater.cc:347] Connected to a master server at 127.15.196.254:44427
I20260929 15:51:05.869051 17044 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:05.869980 17044 heartbeater.cc:511] Master 127.15.196.254:44427 requested a full tablet report, sending...
I20260929 15:51:05.871812 16721 ts_manager.cc:194] Registered new tserver with Master: 15bcc662232a4103b30ca2ae37e1384b (127.15.196.194:33299)
I20260929 15:51:05.873116 16721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:46099
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:51:05.890878 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44427
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:06.215896 17049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:06.216174 17049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:06.216269 17049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:06.223671 17049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:06.223779 17049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:06.223889 17049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:51:06.233743 17049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44427
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:06.235013 17049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:06.236342 17049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:06.248930 17057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:06.249002 17055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:06.249015 17054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:06.249740 17049 server_base.cc:1034] running on GCE node
I20260929 15:51:06.250269 17049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:06.251211 17049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:06.252415 17049 hybrid_clock.cc:648] HybridClock initialized: now 1790697066252375 us; error 31 us; skew 500 ppm
I20260929 15:51:06.252741 17049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:06 dist-test-slave-c51v krb5kdc[16676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697066, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:06.255986 17049 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:51:06.257407 17049 webserver.cc:460] Webserver started at http://127.15.196.195:35047/ using document root <none> and password file <none>
I20260929 15:51:06.258105 17049 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:06.258249 17049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:06.258569 17049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:06.261493 17049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "e13ba129ea6a46659343cb7e718d46b3"
format_stamp: "Formatted at 2026-09-29 15:51:06 on dist-test-slave-c51v"
server_key: "2162f543bf16e4acaf2d8879d53712d8"
server_key_iv: "6c416318b2b569f2f34a8110ac388868"
server_key_version: "encryptionkey@0"
I20260929 15:51:06.262302 17049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "e13ba129ea6a46659343cb7e718d46b3"
format_stamp: "Formatted at 2026-09-29 15:51:06 on dist-test-slave-c51v"
server_key: "2162f543bf16e4acaf2d8879d53712d8"
server_key_iv: "6c416318b2b569f2f34a8110ac388868"
server_key_version: "encryptionkey@0"
I20260929 15:51:06.267606 17049 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 15:51:06.271096 17064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:06.272511 17049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:06.272701 17049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "e13ba129ea6a46659343cb7e718d46b3"
format_stamp: "Formatted at 2026-09-29 15:51:06 on dist-test-slave-c51v"
server_key: "2162f543bf16e4acaf2d8879d53712d8"
server_key_iv: "6c416318b2b569f2f34a8110ac388868"
server_key_version: "encryptionkey@0"
I20260929 15:51:06.272924 17049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:06.293684 17049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:06.298411 17049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:06.298718 17049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:06.299818 17049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:06.301270 17049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:06.301376 17049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:06.301527 17049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:06.301599 17049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:06.334316 17049 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:43971
I20260929 15:51:06.334405 17176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:43971 every 8 connection(s)
I20260929 15:51:06.335681 17049 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:51:06.337423 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 17049
I20260929 15:51:06.337617 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:51:06.338045 16147 external_mini_cluster.cc:1428] Setting key 0b48df69953cce868507a253ff1d38f2
Sep 29 15:51:06 dist-test-slave-c51v krb5kdc[16676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697066, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:06.355741 16912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.338375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:38793 (local address 127.15.196.254:44427)
0929 15:51:06.338585 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:06.338591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:06.339972 (+ 1381us) server_negotiation.cc:407] Connection header received
0929 15:51:06.341902 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:06.341906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:06.341967 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:06.342048 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:06.343489 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.344258 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.345702 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.346090 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.346182 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.349405 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:06.349414 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:06.349419 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:06.349458 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:06.352686 (+ 3228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:06.353505 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:06.353510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:06.353514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:06.353650 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:06.354115 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:06.354118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:06.354120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:06.354426 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:06.354556 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:06.355356 (+ 800us) server_negotiation.cc:299] Negotiation successful
0929 15:51:06.355517 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260929 15:51:06.356981 17179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.338983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44427 (local address 127.15.196.195:38793)
0929 15:51:06.339789 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:06.339851 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:06.341670 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:06.342246 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:06.342254 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:06.342747 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:06.343338 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:06.343363 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.344379 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.344382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:06.345589 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:06.345600 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.346281 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.346285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:06.346474 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.347074 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:06.347100 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:06.349178 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:06.352826 (+ 3648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:06.352836 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:06.352851 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:06.353394 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:06.353759 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:06.353776 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:06.353779 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:06.354005 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:06.354571 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:06.354579 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:06.354948 (+ 369us) client_negotiation.cc:769] Sending connection context
0929 15:51:06.355221 (+ 273us) client_negotiation.cc:240] Negotiation successful
0929 15:51:06.355722 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":373,"threads_started":1}
I20260929 15:51:06.358215 17177 heartbeater.cc:347] Connected to a master server at 127.15.196.254:44427
I20260929 15:51:06.358496 17177 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:06.359159 17177 heartbeater.cc:511] Master 127.15.196.254:44427 requested a full tablet report, sending...
I20260929 15:51:06.360502 16721 ts_manager.cc:194] Registered new tserver with Master: e13ba129ea6a46659343cb7e718d46b3 (127.15.196.195:43971)
I20260929 15:51:06.361263 16721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:38793
I20260929 15:51:06.365437 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:51:06.384043 16912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.371594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55868 (local address 127.15.196.254:44427)
0929 15:51:06.371784 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:06.371803 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:06.372244 (+ 441us) server_negotiation.cc:407] Connection header received
0929 15:51:06.372368 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:06.372372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:06.372427 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:06.372502 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:06.373867 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.374537 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.375996 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.376417 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.376507 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.379134 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:06.379139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:06.379143 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:06.379171 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:06.381674 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:06.382256 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:06.382260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:06.382262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:06.382428 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:06.382772 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:06.382775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:06.382777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:06.383070 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:06.383243 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:06.383604 (+ 361us) server_negotiation.cc:299] Negotiation successful
0929 15:51:06.383751 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 15:51:06.385383 16910 heartbeater.cc:503] Master 127.15.196.254:44427 was elected leader, sending a full tablet report...
I20260929 15:51:06.390285 16721 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:51:06.396937 16721 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:51:06.432312 17190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.408573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54952 (local address 127.15.196.194:33299)
0929 15:51:06.423838 (+ 15265us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:06.423847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:06.423876 (+ 29us) server_negotiation.cc:407] Connection header received
0929 15:51:06.423987 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:06.423994 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:06.424220 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:06.425378 (+ 1158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:06.425610 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.427005 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.429623 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.431043 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.431225 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.431447 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:06.431770 (+ 323us) server_negotiation.cc:299] Negotiation successful
0929 15:51:06.431960 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15110,"thread_start_us":15802,"threads_started":1}
I20260929 15:51:06.432372 17191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.408383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:33299 (local address 127.0.0.1:54952)
0929 15:51:06.422399 (+ 14016us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:06.422448 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:06.422626 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:06.424645 (+ 2019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:06.424650 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:06.424676 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:06.425143 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:06.425157 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.427155 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.427161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:06.429476 (+ 2315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:06.429487 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.431310 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.431318 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:06.431515 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.431568 (+ 53us) client_negotiation.cc:769] Sending connection context
0929 15:51:06.431703 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 15:51:06.431917 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13768,"mutex_wait_us":8,"thread_start_us":14562,"threads_started":1}
I20260929 15:51:06.432821 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.409270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53554 (local address 127.15.196.193:32881)
0929 15:51:06.421585 (+ 12315us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:06.421599 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:06.423360 (+ 1761us) server_negotiation.cc:407] Connection header received
0929 15:51:06.423526 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:06.423536 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:06.423832 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:06.424046 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:06.425016 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.426685 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.429955 (+ 3270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.431385 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.431568 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.431707 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:06.432274 (+ 567us) server_negotiation.cc:299] Negotiation successful
0929 15:51:06.432462 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12055,"thread_start_us":12320,"threads_started":1}
I20260929 15:51:06.433698 17189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.408654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:32881 (local address 127.0.0.1:53554)
0929 15:51:06.423188 (+ 14534us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:06.423205 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:06.423352 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:06.424333 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:06.424343 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:06.424377 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:06.424799 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:06.424814 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.427266 (+ 2452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.427271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:06.429792 (+ 2521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:06.429806 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.431866 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.431872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:06.432063 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.432098 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 15:51:06.433139 (+ 1041us) client_negotiation.cc:240] Negotiation successful
0929 15:51:06.433273 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14423,"mutex_wait_us":157,"thread_start_us":13365,"threads_started":1}
I20260929 15:51:06.434893 17194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.423608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:43971 (local address 127.0.0.1:42594)
0929 15:51:06.427238 (+ 3630us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:06.427254 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:06.427408 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:06.428085 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:06.428091 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:06.428115 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:06.428547 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:06.428559 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.430336 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.430341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:06.431987 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:06.432000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.434172 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.434178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:06.434347 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.434375 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 15:51:06.434471 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 15:51:06.434595 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3477,"thread_start_us":3387,"threads_started":1}
I20260929 15:51:06.435149 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.423210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42594 (local address 127.15.196.195:43971)
0929 15:51:06.426977 (+ 3767us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:06.426988 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:06.427552 (+ 564us) server_negotiation.cc:407] Connection header received
0929 15:51:06.427653 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:06.427660 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:06.427895 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:06.428073 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:06.428746 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.430202 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.432164 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.433701 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.433898 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.434060 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:06.434530 (+ 470us) server_negotiation.cc:299] Negotiation successful
0929 15:51:06.434683 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3587,"thread_start_us":3435,"threads_started":1}
I20260929 15:51:06.437090 16979 tablet_service.cc:1467] Processing CreateTablet for tablet 69bc89fe59fb434eb60c613063c7e5c5 (DEFAULT_TABLE table=test-table [id=3107cd8030254e678c5e22a2e171a550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:06.437316 16845 tablet_service.cc:1467] Processing CreateTablet for tablet 69bc89fe59fb434eb60c613063c7e5c5 (DEFAULT_TABLE table=test-table [id=3107cd8030254e678c5e22a2e171a550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:06.438424 16979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69bc89fe59fb434eb60c613063c7e5c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:06.438722 16845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69bc89fe59fb434eb60c613063c7e5c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:06.438529 17112 tablet_service.cc:1467] Processing CreateTablet for tablet 69bc89fe59fb434eb60c613063c7e5c5 (DEFAULT_TABLE table=test-table [id=3107cd8030254e678c5e22a2e171a550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:06.439870 17112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69bc89fe59fb434eb60c613063c7e5c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:06.449172 17195 tablet_bootstrap.cc:492] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b: Bootstrap starting.
I20260929 15:51:06.452315 17195 tablet_bootstrap.cc:654] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:06.453325 17195 log.cc:824] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:06.454530 17196 tablet_bootstrap.cc:492] T 69bc89fe59fb434eb60c613063c7e5c5 P e13ba129ea6a46659343cb7e718d46b3: Bootstrap starting.
I20260929 15:51:06.458199 17198 tablet_bootstrap.cc:492] T 69bc89fe59fb434eb60c613063c7e5c5 P e4ef1504475c49299790078d1ed114c1: Bootstrap starting.
I20260929 15:51:06.459120 17196 tablet_bootstrap.cc:654] T 69bc89fe59fb434eb60c613063c7e5c5 P e13ba129ea6a46659343cb7e718d46b3: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:06.460608 17196 log.cc:824] T 69bc89fe59fb434eb60c613063c7e5c5 P e13ba129ea6a46659343cb7e718d46b3: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:06.461472 17198 tablet_bootstrap.cc:654] T 69bc89fe59fb434eb60c613063c7e5c5 P e4ef1504475c49299790078d1ed114c1: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:06.462401 17198 log.cc:824] T 69bc89fe59fb434eb60c613063c7e5c5 P e4ef1504475c49299790078d1ed114c1: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:06.463312 17195 tablet_bootstrap.cc:492] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b: No bootstrap required, opened a new log
I20260929 15:51:06.463601 17195 ts_tablet_manager.cc:1397] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.007s
I20260929 15:51:06.464897 17198 tablet_bootstrap.cc:492] T 69bc89fe59fb434eb60c613063c7e5c5 P e4ef1504475c49299790078d1ed114c1: No bootstrap required, opened a new log
I20260929 15:51:06.465134 17198 ts_tablet_manager.cc:1397] T 69bc89fe59fb434eb60c613063c7e5c5 P e4ef1504475c49299790078d1ed114c1: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260929 15:51:06.466904 17196 tablet_bootstrap.cc:492] T 69bc89fe59fb434eb60c613063c7e5c5 P e13ba129ea6a46659343cb7e718d46b3: No bootstrap required, opened a new log
I20260929 15:51:06.467154 17196 ts_tablet_manager.cc:1397] T 69bc89fe59fb434eb60c613063c7e5c5 P e13ba129ea6a46659343cb7e718d46b3: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.005s
I20260929 15:51:06.472894 17198 raft_consensus.cc:348] T 69bc89fe59fb434eb60c613063c7e5c5 P e4ef1504475c49299790078d1ed114c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15bcc662232a4103b30ca2ae37e1384b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33299 } } peers { permanent_uuid: "e4ef1504475c49299790078d1ed114c1" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 32881 } } peers { permanent_uuid: "e13ba129ea6a46659343cb7e718d46b3" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 43971 } }
I20260929 15:51:06.473345 17198 raft_consensus.cc:374] T 69bc89fe59fb434eb60c613063c7e5c5 P e4ef1504475c49299790078d1ed114c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:06.473444 17198 raft_consensus.cc:729] T 69bc89fe59fb434eb60c613063c7e5c5 P e4ef1504475c49299790078d1ed114c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4ef1504475c49299790078d1ed114c1, State: Initialized, Role: FOLLOWER
I20260929 15:51:06.473775 17195 raft_consensus.cc:348] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15bcc662232a4103b30ca2ae37e1384b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33299 } } peers { permanent_uuid: "e4ef1504475c49299790078d1ed114c1" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 32881 } } peers { permanent_uuid: "e13ba129ea6a46659343cb7e718d46b3" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 43971 } }
I20260929 15:51:06.474193 17195 raft_consensus.cc:374] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:06.474061 17198 consensus_queue.cc:260] T 69bc89fe59fb434eb60c613063c7e5c5 P e4ef1504475c49299790078d1ed114c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15bcc662232a4103b30ca2ae37e1384b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33299 } } peers { permanent_uuid: "e4ef1504475c49299790078d1ed114c1" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 32881 } } peers { permanent_uuid: "e13ba129ea6a46659343cb7e718d46b3" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 43971 } }
I20260929 15:51:06.474299 17195 raft_consensus.cc:729] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15bcc662232a4103b30ca2ae37e1384b, State: Initialized, Role: FOLLOWER
I20260929 15:51:06.474857 17195 consensus_queue.cc:260] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15bcc662232a4103b30ca2ae37e1384b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33299 } } peers { permanent_uuid: "e4ef1504475c49299790078d1ed114c1" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 32881 } } peers { permanent_uuid: "e13ba129ea6a46659343cb7e718d46b3" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 43971 } }
I20260929 15:51:06.476150 17044 heartbeater.cc:503] Master 127.15.196.254:44427 was elected leader, sending a full tablet report...
I20260929 15:51:06.476830 17195 ts_tablet_manager.cc:1428] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b: Time spent starting tablet: real 0.013s user 0.002s sys 0.012s
I20260929 15:51:06.477548 17196 raft_consensus.cc:348] T 69bc89fe59fb434eb60c613063c7e5c5 P e13ba129ea6a46659343cb7e718d46b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15bcc662232a4103b30ca2ae37e1384b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33299 } } peers { permanent_uuid: "e4ef1504475c49299790078d1ed114c1" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 32881 } } peers { permanent_uuid: "e13ba129ea6a46659343cb7e718d46b3" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 43971 } }
I20260929 15:51:06.477952 17196 raft_consensus.cc:374] T 69bc89fe59fb434eb60c613063c7e5c5 P e13ba129ea6a46659343cb7e718d46b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:06.478066 17196 raft_consensus.cc:729] T 69bc89fe59fb434eb60c613063c7e5c5 P e13ba129ea6a46659343cb7e718d46b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e13ba129ea6a46659343cb7e718d46b3, State: Initialized, Role: FOLLOWER
I20260929 15:51:06.479249 17198 ts_tablet_manager.cc:1428] T 69bc89fe59fb434eb60c613063c7e5c5 P e4ef1504475c49299790078d1ed114c1: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260929 15:51:06.479677 17196 consensus_queue.cc:260] T 69bc89fe59fb434eb60c613063c7e5c5 P e13ba129ea6a46659343cb7e718d46b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15bcc662232a4103b30ca2ae37e1384b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33299 } } peers { permanent_uuid: "e4ef1504475c49299790078d1ed114c1" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 32881 } } peers { permanent_uuid: "e13ba129ea6a46659343cb7e718d46b3" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 43971 } }
I20260929 15:51:06.481022 17177 heartbeater.cc:503] Master 127.15.196.254:44427 was elected leader, sending a full tablet report...
I20260929 15:51:06.481350 17196 ts_tablet_manager.cc:1428] T 69bc89fe59fb434eb60c613063c7e5c5 P e13ba129ea6a46659343cb7e718d46b3: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260929 15:51:06.536526 17202 raft_consensus.cc:482] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:51:06.536849 17202 raft_consensus.cc:504] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15bcc662232a4103b30ca2ae37e1384b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33299 } } peers { permanent_uuid: "e4ef1504475c49299790078d1ed114c1" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 32881 } } peers { permanent_uuid: "e13ba129ea6a46659343cb7e718d46b3" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 43971 } }
I20260929 15:51:06.538564 17202 leader_election.cc:288] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4ef1504475c49299790078d1ed114c1 (127.15.196.193:32881), e13ba129ea6a46659343cb7e718d46b3 (127.15.196.195:43971)
I20260929 15:51:06.545674 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.539055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:39615 (local address 127.15.196.193:32881)
0929 15:51:06.539356 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:06.539364 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:06.539599 (+ 235us) server_negotiation.cc:407] Connection header received
0929 15:51:06.539815 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:06.539820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:06.539913 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:06.540008 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:06.540764 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.541760 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.543779 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.544652 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.544810 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.544900 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:06.545272 (+ 372us) server_negotiation.cc:299] Negotiation successful
0929 15:51:06.545408 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 15:51:06.545717 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.539915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:47827 (local address 127.15.196.195:43971)
0929 15:51:06.540113 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:06.540120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:06.540324 (+ 204us) server_negotiation.cc:407] Connection header received
0929 15:51:06.540462 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:06.540468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:06.540551 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:06.540656 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:06.541335 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.542199 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.543952 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.544696 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.544893 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.544968 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:06.545306 (+ 338us) server_negotiation.cc:299] Negotiation successful
0929 15:51:06.545426 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 15:51:06.545877 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.539661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:43971 (local address 127.15.196.194:47827)
0929 15:51:06.540227 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:06.540241 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:06.540357 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:06.540764 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:06.540769 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:06.540794 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:06.541166 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:06.541177 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.542336 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.542339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:06.543799 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:06.543812 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.544955 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.544960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:06.545130 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.545152 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 15:51:06.545260 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 15:51:06.545415 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":28,"thread_start_us":272,"threads_started":1}
I20260929 15:51:06.545903 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.538798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:32881 (local address 127.15.196.194:39615)
0929 15:51:06.539494 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:06.539511 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:06.539673 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:06.540156 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:06.540161 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:06.540191 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:06.540590 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:06.540605 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.541896 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.541901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:06.543607 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:06.543618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.544893 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.544900 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:06.545110 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.545134 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 15:51:06.545208 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 15:51:06.545379 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":28,"thread_start_us":348,"threads_started":1}
I20260929 15:51:06.547212 17132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bc89fe59fb434eb60c613063c7e5c5" candidate_uuid: "15bcc662232a4103b30ca2ae37e1384b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e13ba129ea6a46659343cb7e718d46b3" is_pre_election: true
I20260929 15:51:06.547212 16865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bc89fe59fb434eb60c613063c7e5c5" candidate_uuid: "15bcc662232a4103b30ca2ae37e1384b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4ef1504475c49299790078d1ed114c1" is_pre_election: true
I20260929 15:51:06.547827 16865 raft_consensus.cc:2454] T 69bc89fe59fb434eb60c613063c7e5c5 P e4ef1504475c49299790078d1ed114c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15bcc662232a4103b30ca2ae37e1384b in term 0.
I20260929 15:51:06.547832 17132 raft_consensus.cc:2454] T 69bc89fe59fb434eb60c613063c7e5c5 P e13ba129ea6a46659343cb7e718d46b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15bcc662232a4103b30ca2ae37e1384b in term 0.
I20260929 15:51:06.548508 16932 leader_election.cc:302] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15bcc662232a4103b30ca2ae37e1384b, e13ba129ea6a46659343cb7e718d46b3; no voters:
I20260929 15:51:06.548856 17202 raft_consensus.cc:2781] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:51:06.549005 17202 raft_consensus.cc:482] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:51:06.549108 17202 raft_consensus.cc:3037] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:06.551518 17202 raft_consensus.cc:504] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15bcc662232a4103b30ca2ae37e1384b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33299 } } peers { permanent_uuid: "e4ef1504475c49299790078d1ed114c1" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 32881 } } peers { permanent_uuid: "e13ba129ea6a46659343cb7e718d46b3" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 43971 } }
I20260929 15:51:06.552114 17202 leader_election.cc:288] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [CANDIDATE]: Term 1 election: Requested vote from peers e4ef1504475c49299790078d1ed114c1 (127.15.196.193:32881), e13ba129ea6a46659343cb7e718d46b3 (127.15.196.195:43971)
I20260929 15:51:06.552510 16865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bc89fe59fb434eb60c613063c7e5c5" candidate_uuid: "15bcc662232a4103b30ca2ae37e1384b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4ef1504475c49299790078d1ed114c1"
I20260929 15:51:06.552603 17132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bc89fe59fb434eb60c613063c7e5c5" candidate_uuid: "15bcc662232a4103b30ca2ae37e1384b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e13ba129ea6a46659343cb7e718d46b3"
I20260929 15:51:06.552701 16865 raft_consensus.cc:3037] T 69bc89fe59fb434eb60c613063c7e5c5 P e4ef1504475c49299790078d1ed114c1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:06.552848 17132 raft_consensus.cc:3037] T 69bc89fe59fb434eb60c613063c7e5c5 P e13ba129ea6a46659343cb7e718d46b3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:06.554936 16865 raft_consensus.cc:2454] T 69bc89fe59fb434eb60c613063c7e5c5 P e4ef1504475c49299790078d1ed114c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15bcc662232a4103b30ca2ae37e1384b in term 1.
I20260929 15:51:06.555030 17132 raft_consensus.cc:2454] T 69bc89fe59fb434eb60c613063c7e5c5 P e13ba129ea6a46659343cb7e718d46b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15bcc662232a4103b30ca2ae37e1384b in term 1.
I20260929 15:51:06.555387 16932 leader_election.cc:302] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15bcc662232a4103b30ca2ae37e1384b, e4ef1504475c49299790078d1ed114c1; no voters:
I20260929 15:51:06.555646 17202 raft_consensus.cc:2781] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:06.556089 17202 raft_consensus.cc:686] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [term 1 LEADER]: Becoming Leader. State: Replica: 15bcc662232a4103b30ca2ae37e1384b, State: Running, Role: LEADER
I20260929 15:51:06.556819 17202 consensus_queue.cc:237] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15bcc662232a4103b30ca2ae37e1384b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33299 } } peers { permanent_uuid: "e4ef1504475c49299790078d1ed114c1" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 32881 } } peers { permanent_uuid: "e13ba129ea6a46659343cb7e718d46b3" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 43971 } }
I20260929 15:51:06.561446 16719 catalog_manager.cc:5495] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b reported cstate change: term changed from 0 to 1, leader changed from <none> to 15bcc662232a4103b30ca2ae37e1384b (127.15.196.194). New cstate: current_term: 1 leader_uuid: "15bcc662232a4103b30ca2ae37e1384b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15bcc662232a4103b30ca2ae37e1384b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4ef1504475c49299790078d1ed114c1" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 32881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e13ba129ea6a46659343cb7e718d46b3" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 43971 } health_report { overall_health: UNKNOWN } } }
W20260929 15:51:06.587527 17178 tablet.cc:2369] T 69bc89fe59fb434eb60c613063c7e5c5 P e13ba129ea6a46659343cb7e718d46b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 15:51:06.595822 17045 tablet.cc:2369] T 69bc89fe59fb434eb60c613063c7e5c5 P 15bcc662232a4103b30ca2ae37e1384b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 15:51:06.604733 16911 tablet.cc:2369] T 69bc89fe59fb434eb60c613063c7e5c5 P e4ef1504475c49299790078d1ed114c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 15:51:06 dist-test-slave-c51v krb5kdc[16676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:51:06.615556 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.601921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53566 (local address 127.15.196.193:32881)
0929 15:51:06.602086 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:06.602091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:06.602587 (+ 496us) server_negotiation.cc:407] Connection header received
0929 15:51:06.602718 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:06.602721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:06.602788 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:06.602872 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:06.604333 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.605160 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.606437 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.606854 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.606945 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.610666 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:06.610678 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:06.610693 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:06.610727 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:06.613287 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:06.613864 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:06.613871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:06.613873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:06.613997 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:06.614284 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:06.614287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:06.614289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:06.614629 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:06.614858 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:06.615086 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 15:51:06.615236 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 15:51:06 dist-test-slave-c51v krb5kdc[16676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697066, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 15:51:06 dist-test-slave-c51v krb5kdc[16676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697066, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:51:06.659360 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.645938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53578 (local address 127.15.196.193:32881)
0929 15:51:06.646164 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:06.646170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:06.646428 (+ 258us) server_negotiation.cc:407] Connection header received
0929 15:51:06.646574 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:06.646578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:06.646643 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:06.646729 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:06.648043 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.648754 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.649964 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.650413 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.650556 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.654589 (+ 4033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:06.654594 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:06.654597 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:06.654628 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:06.657209 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:06.657781 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:06.657785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:06.657787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:06.657905 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:06.658252 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:06.658254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:06.658256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:06.658499 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:06.658688 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:06.658883 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 15:51:06.659042 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Sep 29 15:51:06 dist-test-slave-c51v krb5kdc[16676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 15:51:06 dist-test-slave-c51v krb5kdc[16676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:51:06.702682 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.689023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53584 (local address 127.15.196.193:32881)
0929 15:51:06.689282 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:06.689286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:06.689382 (+ 96us) server_negotiation.cc:407] Connection header received
0929 15:51:06.689532 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:06.689536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:06.689605 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:06.689708 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:06.691033 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.691951 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.693146 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.693616 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.693723 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.697592 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:06.697597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:06.697601 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:06.697645 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:06.700029 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:06.700727 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:06.700734 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:06.700738 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:06.700864 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:06.701226 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:06.701231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:06.701234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:06.701534 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:06.701799 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:06.702087 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 15:51:06.702332 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260929 15:51:06.704493 16825 tablet_service.cc:2908] Rejecting scan request for tablet 69bc89fe59fb434eb60c613063c7e5c5: Uninitialized: clean time has not yet been initialized
I20260929 15:51:06.707458 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 16782
I20260929 15:51:06.719214 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.708371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53596 (local address 127.15.196.193:32881)
0929 15:51:06.708533 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:06.708537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:06.708911 (+ 374us) server_negotiation.cc:407] Connection header received
0929 15:51:06.709042 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:06.709046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:06.709101 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:06.709180 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:06.710279 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.710952 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.712122 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.712540 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.712642 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.714738 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:06.714743 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:06.714745 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:06.714774 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:06.717176 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:06.717781 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:06.717785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:06.717787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:06.717905 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:06.718188 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:06.718191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:06.718192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:06.718431 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:06.718631 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:06.718831 (+ 200us) server_negotiation.cc:299] Negotiation successful
0929 15:51:06.718994 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 15:51:06.720105 16905 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:06.809862 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 16782
I20260929 15:51:06.826473 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 16916
Sep 29 15:51:06 dist-test-slave-c51v krb5kdc[16676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:51:06.840471 17190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.827196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54966 (local address 127.15.196.194:33299)
0929 15:51:06.827451 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:06.827455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:06.827473 (+ 18us) server_negotiation.cc:407] Connection header received
0929 15:51:06.827593 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:06.827596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:06.827656 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:06.827736 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:06.828841 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.829530 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.830719 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.831097 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.831184 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.835265 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:06.835278 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:06.835355 (+ 77us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:06.835413 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:06.838025 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:06.838636 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:06.838643 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:06.838645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:06.838779 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:06.839080 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:06.839083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:06.839085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:06.839496 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:06.839708 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:06.839947 (+ 239us) server_negotiation.cc:299] Negotiation successful
0929 15:51:06.840114 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 15:51:06.841151 17039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:06.930313 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 16916
I20260929 15:51:06.948473 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 17049
Sep 29 15:51:06 dist-test-slave-c51v krb5kdc[16676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:51:06.962949 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:06.949265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42606 (local address 127.15.196.195:43971)
0929 15:51:06.949439 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:06.949444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:06.949488 (+ 44us) server_negotiation.cc:407] Connection header received
0929 15:51:06.949708 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:06.949715 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:06.949801 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:06.949896 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:06.951533 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.952256 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.953474 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:06.953861 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:06.953949 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:06.957918 (+ 3969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:06.957933 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:06.957948 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:06.957985 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:06.960732 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:06.961275 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:06.961281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:06.961283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:06.961402 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:06.961706 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:06.961709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:06.961711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:06.962040 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:06.962249 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:06.962435 (+ 186us) server_negotiation.cc:299] Negotiation successful
0929 15:51:06.962584 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 15:51:06.963629 17172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:07.054335 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 17049
I20260929 15:51:07.070534 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 16689
I20260929 15:51:07.071658 16751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:07.178637 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 16689
2026-09-29T15:51:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4101 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:51:07 dist-test-slave-c51v krb5kdc[17225](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:51:07 dist-test-slave-c51v krb5kdc[17225](info): set up 2 sockets
Sep 29 15:51:07 dist-test-slave-c51v krb5kdc[17225](info): commencing operation
krb5kdc: starting...
W20260929 15:51:08.265441 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:51:08 dist-test-slave-c51v krb5kdc[17225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:51:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:51:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:51:08.443261 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:35441
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:35441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:08.758201 17238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:08.758500 17238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:08.758654 17238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:08.766141 17238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:08.766263 17238 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 15:51:08.766341 17238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:08.766458 17238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:08.766536 17238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:08.766649 17238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:51:08.776726 17238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:35441
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:35441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:08.778162 17238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:08.779565 17238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:08.791723 17243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:08.791875 17246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:08.792060 17244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:08.792649 17238 server_base.cc:1034] running on GCE node
I20260929 15:51:08.793452 17238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:08.794937 17238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:08.796180 17238 hybrid_clock.cc:648] HybridClock initialized: now 1790697068796115 us; error 42 us; skew 500 ppm
I20260929 15:51:08.796525 17238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:08 dist-test-slave-c51v krb5kdc[17225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697068, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:08.799659 17238 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:51:08.801193 17238 webserver.cc:460] Webserver started at http://127.15.196.254:42467/ using document root <none> and password file <none>
I20260929 15:51:08.801916 17238 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:08.802044 17238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:08.802371 17238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:08.805332 17238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "032855516d3642db8cde6bf44d13c1ed"
format_stamp: "Formatted at 2026-09-29 15:51:08 on dist-test-slave-c51v"
server_key: "0247bceae5ecddcff206fe4920bd3437"
server_key_iv: "601376d0d4b964a7a570d380c0477d0b"
server_key_version: "encryptionkey@0"
I20260929 15:51:08.806125 17238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "032855516d3642db8cde6bf44d13c1ed"
format_stamp: "Formatted at 2026-09-29 15:51:08 on dist-test-slave-c51v"
server_key: "0247bceae5ecddcff206fe4920bd3437"
server_key_iv: "601376d0d4b964a7a570d380c0477d0b"
server_key_version: "encryptionkey@0"
I20260929 15:51:08.811228 17238 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 15:51:08.814539 17253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:08.815881 17238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:08.816058 17238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "032855516d3642db8cde6bf44d13c1ed"
format_stamp: "Formatted at 2026-09-29 15:51:08 on dist-test-slave-c51v"
server_key: "0247bceae5ecddcff206fe4920bd3437"
server_key_iv: "601376d0d4b964a7a570d380c0477d0b"
server_key_version: "encryptionkey@0"
I20260929 15:51:08.816274 17238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:08.851807 17238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:08.855958 17238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:08.856271 17238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:08.878832 17238 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:35441
I20260929 15:51:08.878861 17314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:35441 every 8 connection(s)
I20260929 15:51:08.880311 17238 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:51:08.884393 17315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:08.890590 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 17238
I20260929 15:51:08.890853 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:51:08.891578 16147 external_mini_cluster.cc:1428] Setting key 286d96c0cfc6f7e5d82cd4630a971e1d
I20260929 15:51:08.895263 17315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed: Bootstrap starting.
I20260929 15:51:08.898870 17315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:08.900309 17315 log.cc:824] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:08.903498 17315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed: No bootstrap required, opened a new log
Sep 29 15:51:08 dist-test-slave-c51v krb5kdc[17225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:08.913401 17315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032855516d3642db8cde6bf44d13c1ed" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35441 } }
I20260929 15:51:08.913872 17315 raft_consensus.cc:374] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:08.913991 17315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 032855516d3642db8cde6bf44d13c1ed, State: Initialized, Role: FOLLOWER
I20260929 15:51:08.914700 17315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032855516d3642db8cde6bf44d13c1ed" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35441 } }
I20260929 15:51:08.915021 17315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:51:08.915181 17315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:51:08.915418 17315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:08.916787 17318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:08.893536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48368 (local address 127.15.196.254:35441)
0929 15:51:08.894322 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:08.894334 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:08.894376 (+ 42us) server_negotiation.cc:407] Connection header received
0929 15:51:08.896407 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:08.896431 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:08.896888 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:08.897242 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:08.899098 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:08.901136 (+ 2038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:08.902477 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:08.903513 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:08.903706 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:08.907792 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:08.907804 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:08.907826 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:08.907895 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:08.912018 (+ 4123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:08.912700 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:08.912708 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:08.912717 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:08.912920 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:08.913257 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:08.913261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:08.913264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:08.914033 (+ 769us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:08.914331 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:08.914809 (+ 478us) server_negotiation.cc:299] Negotiation successful
0929 15:51:08.915306 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":370,"threads_started":1}
I20260929 15:51:08.918262 17315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032855516d3642db8cde6bf44d13c1ed" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35441 } }
I20260929 15:51:08.918781 17315 leader_election.cc:302] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 032855516d3642db8cde6bf44d13c1ed; no voters:
I20260929 15:51:08.919394 17315 leader_election.cc:288] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:51:08.919590 17320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:08.920540 17320 raft_consensus.cc:686] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [term 1 LEADER]: Becoming Leader. State: Replica: 032855516d3642db8cde6bf44d13c1ed, State: Running, Role: LEADER
I20260929 15:51:08.921134 17320 consensus_queue.cc:237] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032855516d3642db8cde6bf44d13c1ed" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35441 } }
I20260929 15:51:08.921797 17315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:08.924196 17321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "032855516d3642db8cde6bf44d13c1ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032855516d3642db8cde6bf44d13c1ed" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35441 } } }
I20260929 15:51:08.924556 17321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:08.925410 17322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 032855516d3642db8cde6bf44d13c1ed. Latest consensus state: current_term: 1 leader_uuid: "032855516d3642db8cde6bf44d13c1ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032855516d3642db8cde6bf44d13c1ed" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35441 } } }
I20260929 15:51:08.925855 17322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:08.928215 17327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:08.933955 17327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:08.946173 17327 catalog_manager.cc:1347] Generated new cluster ID: d2247277e82e49ac996a6598e1ee7dae
I20260929 15:51:08.946287 17327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:08.963398 17327 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:51:08.965272 17327 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:08.982358 17327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed: Generated new TSK 0
I20260929 15:51:08.983498 17327 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:51:09.037279 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:35441
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:09.379043 17341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:09.379452 17341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:09.379662 17341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:09.387569 17341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:09.387737 17341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:09.387910 17341 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 15:51:09.388049 17341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:51:09.398571 17341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.196.254:35441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:09.400029 17341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:09.401398 17341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:09.415652 17346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:09.415684 17347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:09.415917 17349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:09.416857 17341 server_base.cc:1034] running on GCE node
I20260929 15:51:09.417779 17341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:09.418838 17341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:09.420096 17341 hybrid_clock.cc:648] HybridClock initialized: now 1790697069420033 us; error 42 us; skew 500 ppm
I20260929 15:51:09.420430 17341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:09 dist-test-slave-c51v krb5kdc[17225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697069, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:09.423815 17341 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:51:09.425275 17341 webserver.cc:460] Webserver started at http://127.15.196.193:45681/ using document root <none> and password file <none>
I20260929 15:51:09.426281 17341 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:09.426435 17341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:09.426819 17341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:09.429805 17341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "785a4dfd9aa34d3289e50ab9647ecf8e"
format_stamp: "Formatted at 2026-09-29 15:51:09 on dist-test-slave-c51v"
server_key: "bdb122bf4191482c51ba4229368420f0"
server_key_iv: "c2aca7ad1c8713042d17974b70f77beb"
server_key_version: "encryptionkey@0"
I20260929 15:51:09.430572 17341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "785a4dfd9aa34d3289e50ab9647ecf8e"
format_stamp: "Formatted at 2026-09-29 15:51:09 on dist-test-slave-c51v"
server_key: "bdb122bf4191482c51ba4229368420f0"
server_key_iv: "c2aca7ad1c8713042d17974b70f77beb"
server_key_version: "encryptionkey@0"
I20260929 15:51:09.435810 17341 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260929 15:51:09.439241 17356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:09.440557 17341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:51:09.440739 17341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "785a4dfd9aa34d3289e50ab9647ecf8e"
format_stamp: "Formatted at 2026-09-29 15:51:09 on dist-test-slave-c51v"
server_key: "bdb122bf4191482c51ba4229368420f0"
server_key_iv: "c2aca7ad1c8713042d17974b70f77beb"
server_key_version: "encryptionkey@0"
I20260929 15:51:09.440958 17341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:09.457909 17341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:09.461807 17341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:09.462073 17341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:09.464846 17341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:09.464952 17341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:09.465132 17341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:09.465291 17341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Sep 29 15:51:09 dist-test-slave-c51v krb5kdc[17225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697069, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:09.488323 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:09.465348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:59169 (local address 127.15.196.254:35441)
0929 15:51:09.465839 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:09.465845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:09.467005 (+ 1160us) server_negotiation.cc:407] Connection header received
0929 15:51:09.469221 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:09.469229 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:09.469328 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:09.469446 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:09.471474 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:09.472499 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:09.475073 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:09.475682 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:09.475819 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:09.481830 (+ 6011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:09.481838 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:09.481843 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:09.481885 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:09.485028 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:09.485810 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:09.485816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:09.485820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:09.485941 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:09.486449 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:09.486454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:09.486458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:09.486737 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:09.486893 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:09.487712 (+ 819us) server_negotiation.cc:299] Negotiation successful
0929 15:51:09.487943 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":242,"threads_started":1}
I20260929 15:51:09.489866 17366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:09.465926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:35441 (local address 127.15.196.193:59169)
0929 15:51:09.466811 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:09.466859 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:09.468939 (+ 2080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:09.469677 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:09.469685 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:09.470358 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:09.471245 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:09.471265 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:09.472663 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:09.472668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:09.474835 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:09.474848 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:09.476013 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:09.476023 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:09.476362 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:09.477297 (+ 935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:09.477341 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:09.481550 (+ 4209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:09.485219 (+ 3669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:09.485231 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:09.485249 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:09.485675 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:09.486052 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:09.486057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:09.486061 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:09.486319 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:09.486872 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:09.486880 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:09.487269 (+ 389us) client_negotiation.cc:769] Sending connection context
0929 15:51:09.487627 (+ 358us) client_negotiation.cc:240] Negotiation successful
0929 15:51:09.488198 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":362,"threads_started":1}
I20260929 15:51:09.509610 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:09.496275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:45239 (local address 127.15.196.254:35441)
0929 15:51:09.496496 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:09.496504 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:09.497055 (+ 551us) server_negotiation.cc:407] Connection header received
0929 15:51:09.497253 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:09.497260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:09.497330 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:09.497445 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:09.498354 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:09.499388 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:09.500661 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:09.501058 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:09.501201 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:09.502955 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:09.502961 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:09.502966 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:09.503006 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:09.505938 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:09.506790 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:09.506800 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:09.506807 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:09.507009 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:09.507527 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:09.507534 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:09.507538 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:09.507911 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:09.508066 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:09.509105 (+ 1039us) server_negotiation.cc:299] Negotiation successful
0929 15:51:09.509356 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 15:51:09.509702 17423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:09.496241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:35441 (local address 127.15.196.193:45239)
0929 15:51:09.496950 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:09.496967 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:09.497125 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:09.497480 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:09.497485 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:09.497940 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:09.498215 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:09.498224 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:09.499530 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:09.499534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:09.500524 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:09.500535 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:09.501245 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:09.501252 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:09.501437 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:09.502093 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:09.502110 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:09.502779 (+ 669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:09.506073 (+ 3294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:09.506078 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:09.506082 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:09.506604 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:09.507164 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:09.507171 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:09.507176 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:09.507402 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:09.508126 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:09.508129 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:09.508349 (+ 220us) client_negotiation.cc:769] Sending connection context
0929 15:51:09.508555 (+ 206us) client_negotiation.cc:240] Negotiation successful
0929 15:51:09.508880 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":477,"threads_started":1}
I20260929 15:51:09.521059 17341 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:35757
I20260929 15:51:09.521122 17476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:35757 every 8 connection(s)
I20260929 15:51:09.522403 17341 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:51:09.524525 17477 heartbeater.cc:347] Connected to a master server at 127.15.196.254:35441
I20260929 15:51:09.524725 17477 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:09.525326 17477 heartbeater.cc:511] Master 127.15.196.254:35441 requested a full tablet report, sending...
I20260929 15:51:09.527273 17270 ts_manager.cc:194] Registered new tserver with Master: 785a4dfd9aa34d3289e50ab9647ecf8e (127.15.196.193:35757)
I20260929 15:51:09.527563 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 17341
I20260929 15:51:09.527787 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:51:09.528610 16147 external_mini_cluster.cc:1428] Setting key 979b08956bbb62067b9068031cae0ada
I20260929 15:51:09.529349 17270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:59169
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:51:09.566712 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:35441
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:09.880990 17481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:09.881243 17481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:09.881407 17481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:09.888533 17481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:09.888635 17481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:09.888775 17481 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 15:51:09.888888 17481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:51:09.898552 17481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.196.254:35441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:09.899883 17481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:09.901249 17481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:09.914507 17486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:09.914566 17487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:09.914589 17489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:09.915108 17481 server_base.cc:1034] running on GCE node
I20260929 15:51:09.915810 17481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:09.916730 17481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:09.917971 17481 hybrid_clock.cc:648] HybridClock initialized: now 1790697069917910 us; error 37 us; skew 500 ppm
I20260929 15:51:09.918375 17481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:09 dist-test-slave-c51v krb5kdc[17225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697069, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:09.924546 17481 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:51:09.926020 17481 webserver.cc:460] Webserver started at http://127.15.196.194:33419/ using document root <none> and password file <none>
I20260929 15:51:09.926715 17481 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:09.926847 17481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:09.927170 17481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:09.930073 17481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "138c5c0fd7024a288fc85c1d52cf71d0"
format_stamp: "Formatted at 2026-09-29 15:51:09 on dist-test-slave-c51v"
server_key: "3ea1f63f5af4d7632395cad189538935"
server_key_iv: "7665d8da1064b222e01bcd03a14a3fff"
server_key_version: "encryptionkey@0"
I20260929 15:51:09.930940 17481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "138c5c0fd7024a288fc85c1d52cf71d0"
format_stamp: "Formatted at 2026-09-29 15:51:09 on dist-test-slave-c51v"
server_key: "3ea1f63f5af4d7632395cad189538935"
server_key_iv: "7665d8da1064b222e01bcd03a14a3fff"
server_key_version: "encryptionkey@0"
I20260929 15:51:09.936199 17481 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260929 15:51:09.939805 17496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:09.941229 17481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:51:09.941398 17481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "138c5c0fd7024a288fc85c1d52cf71d0"
format_stamp: "Formatted at 2026-09-29 15:51:09 on dist-test-slave-c51v"
server_key: "3ea1f63f5af4d7632395cad189538935"
server_key_iv: "7665d8da1064b222e01bcd03a14a3fff"
server_key_version: "encryptionkey@0"
I20260929 15:51:09.941603 17481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:09.969151 17481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:09.973598 17481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:09.973938 17481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:09.976943 17481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:09.977077 17481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:09.977249 17481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:09.977336 17481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 15:51:09 dist-test-slave-c51v krb5kdc[17225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697069, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:10.001107 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:09.977354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:59321 (local address 127.15.196.254:35441)
0929 15:51:09.977577 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:09.977584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:09.979051 (+ 1467us) server_negotiation.cc:407] Connection header received
0929 15:51:09.981053 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:09.981059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:09.981190 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:09.981304 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:09.983613 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:09.985064 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:09.987628 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:09.988070 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:09.988166 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:09.993831 (+ 5665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:09.993836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:09.993840 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:09.993876 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:09.997177 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:09.998168 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:09.998179 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:09.998181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:09.998302 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:09.998876 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:09.998881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:09.998884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:09.999282 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:09.999485 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:10.000494 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0929 15:51:10.000752 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 15:51:10.002240 17505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:09.977964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:35441 (local address 127.15.196.194:59321)
0929 15:51:09.978845 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:09.978902 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:09.980764 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:09.981600 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:09.981613 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:09.982409 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:09.983267 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:09.983347 (+ 80us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:09.985262 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:09.985266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:09.987422 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:09.987440 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:09.988346 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:09.988358 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:09.988793 (+ 435us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:09.989806 (+ 1013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:09.989854 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:09.993561 (+ 3707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:09.997344 (+ 3783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:09.997372 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:09.997400 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:09.998017 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:09.998419 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:09.998424 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:09.998428 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:09.998753 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:09.999557 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:09.999564 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:10.000099 (+ 535us) client_negotiation.cc:769] Sending connection context
0929 15:51:10.000402 (+ 303us) client_negotiation.cc:240] Negotiation successful
0929 15:51:10.000969 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":387,"threads_started":1}
I20260929 15:51:10.024887 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.008463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:35441 (local address 127.15.196.194:38687)
0929 15:51:10.009169 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:10.009185 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:10.009353 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:10.009698 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:10.009703 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:10.010367 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:10.010755 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:10.010764 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.012207 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.012214 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:10.013885 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:10.013907 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.014980 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.014984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:10.015101 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.015898 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:10.015914 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:10.017087 (+ 1173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:10.021821 (+ 4734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:10.021825 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:10.021828 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:10.022269 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:10.022807 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:10.022811 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:10.022814 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:10.022966 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:10.023860 (+ 894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:10.023869 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:10.024121 (+ 252us) client_negotiation.cc:769] Sending connection context
0929 15:51:10.024278 (+ 157us) client_negotiation.cc:240] Negotiation successful
0929 15:51:10.024533 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":460,"threads_started":1}
I20260929 15:51:10.025081 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.008271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:38687 (local address 127.15.196.254:35441)
0929 15:51:10.008422 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:10.008426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:10.009277 (+ 851us) server_negotiation.cc:407] Connection header received
0929 15:51:10.009504 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:10.009508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:10.009570 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:10.009671 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:10.010989 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.011997 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.014084 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.014653 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.014754 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.017372 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:10.017377 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:10.017380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:10.017410 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:10.021667 (+ 4257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:10.022433 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:10.022442 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:10.022449 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:10.022669 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:10.023081 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:10.023089 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:10.023095 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:10.023690 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:10.023877 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:10.024426 (+ 549us) server_negotiation.cc:299] Negotiation successful
0929 15:51:10.024688 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 15:51:10.039369 17481 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:34397
I20260929 15:51:10.039383 17615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:34397 every 8 connection(s)
I20260929 15:51:10.040767 17481 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:51:10.042852 17616 heartbeater.cc:347] Connected to a master server at 127.15.196.254:35441
I20260929 15:51:10.042981 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 17481
I20260929 15:51:10.043076 17616 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:10.043229 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:51:10.043740 16147 external_mini_cluster.cc:1428] Setting key 148bdc1570defd4909bfe0fba379a31f
I20260929 15:51:10.043804 17616 heartbeater.cc:511] Master 127.15.196.254:35441 requested a full tablet report, sending...
I20260929 15:51:10.045367 17270 ts_manager.cc:194] Registered new tserver with Master: 138c5c0fd7024a288fc85c1d52cf71d0 (127.15.196.194:34397)
I20260929 15:51:10.046577 17270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:59321
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:51:10.083235 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:35441
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:10.404871 17620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:10.405203 17620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:10.405383 17620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:10.412853 17620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:10.413017 17620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:10.413156 17620 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 15:51:10.413288 17620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:51:10.423677 17620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.196.254:35441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:10.425051 17620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:10.426406 17620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:10.439749 17626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:10.439775 17628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:10.439806 17625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:10.440617 17620 server_base.cc:1034] running on GCE node
I20260929 15:51:10.441342 17620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:10.442240 17620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:10.443470 17620 hybrid_clock.cc:648] HybridClock initialized: now 1790697070443400 us; error 41 us; skew 500 ppm
I20260929 15:51:10.443766 17620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:10 dist-test-slave-c51v krb5kdc[17225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697070, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:10.446974 17620 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:51:10.448566 17620 webserver.cc:460] Webserver started at http://127.15.196.195:34083/ using document root <none> and password file <none>
I20260929 15:51:10.449220 17620 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:10.449303 17620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:10.449568 17620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:10.452382 17620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "8c8fcc190371414fbc3f9dc19f539335"
format_stamp: "Formatted at 2026-09-29 15:51:10 on dist-test-slave-c51v"
server_key: "24370de724a2a90281081511092723d1"
server_key_iv: "8b095a628b9484b10ca9542af4d8ce81"
server_key_version: "encryptionkey@0"
I20260929 15:51:10.453106 17620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "8c8fcc190371414fbc3f9dc19f539335"
format_stamp: "Formatted at 2026-09-29 15:51:10 on dist-test-slave-c51v"
server_key: "24370de724a2a90281081511092723d1"
server_key_iv: "8b095a628b9484b10ca9542af4d8ce81"
server_key_version: "encryptionkey@0"
I20260929 15:51:10.457973 17620 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:51:10.461220 17635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:10.462502 17620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:51:10.462632 17620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "8c8fcc190371414fbc3f9dc19f539335"
format_stamp: "Formatted at 2026-09-29 15:51:10 on dist-test-slave-c51v"
server_key: "24370de724a2a90281081511092723d1"
server_key_iv: "8b095a628b9484b10ca9542af4d8ce81"
server_key_version: "encryptionkey@0"
I20260929 15:51:10.462803 17620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:10.506862 17620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:10.510898 17620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:10.511114 17620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:10.513648 17620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:10.513772 17620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:10.513931 17620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:10.514004 17620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 15:51:10 dist-test-slave-c51v krb5kdc[17225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697070, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:10.532665 17477 heartbeater.cc:503] Master 127.15.196.254:35441 was elected leader, sending a full tablet report...
I20260929 15:51:10.539503 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.514418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:38111 (local address 127.15.196.254:35441)
0929 15:51:10.514612 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:10.514618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:10.516145 (+ 1527us) server_negotiation.cc:407] Connection header received
0929 15:51:10.518123 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:10.518129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:10.518216 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:10.518335 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:10.520611 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.521622 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.524978 (+ 3356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.525720 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.525870 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.530176 (+ 4306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:10.530182 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:10.530189 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:10.530228 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:10.533897 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:10.536172 (+ 2275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:10.536181 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:10.536187 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:10.536385 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:10.537134 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:10.537138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:10.537140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:10.537596 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:10.537844 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:10.538779 (+ 935us) server_negotiation.cc:299] Negotiation successful
0929 15:51:10.539041 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 15:51:10.540922 17644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.515047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:35441 (local address 127.15.196.195:38111)
0929 15:51:10.515877 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:10.515923 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:10.517850 (+ 1927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:10.518611 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:10.518629 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:10.519460 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:10.520421 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:10.520440 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.521769 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.521777 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:10.524755 (+ 2978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:10.524775 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.525978 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.525986 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:10.526265 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.527068 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:10.527100 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:10.529949 (+ 2849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:10.535046 (+ 5097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:10.535055 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:10.535071 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:10.535890 (+ 819us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:10.536562 (+ 672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:10.536567 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:10.536574 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:10.536948 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:10.537814 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:10.537827 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:10.538267 (+ 440us) client_negotiation.cc:769] Sending connection context
0929 15:51:10.538604 (+ 337us) client_negotiation.cc:240] Negotiation successful
0929 15:51:10.539274 (+ 670us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":381,"threads_started":1}
I20260929 15:51:10.559875 17620 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:41927
I20260929 15:51:10.559893 17754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:41927 every 8 connection(s)
I20260929 15:51:10.561174 17620 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:51:10.563560 17755 heartbeater.cc:347] Connected to a master server at 127.15.196.254:35441
I20260929 15:51:10.563890 17755 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:10.564662 17755 heartbeater.cc:511] Master 127.15.196.254:35441 requested a full tablet report, sending...
I20260929 15:51:10.566097 17270 ts_manager.cc:194] Registered new tserver with Master: 8c8fcc190371414fbc3f9dc19f539335 (127.15.196.195:41927)
I20260929 15:51:10.566449 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.549078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:35441 (local address 127.15.196.195:35739)
0929 15:51:10.549896 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:10.549931 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:10.550137 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:10.550527 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:10.550533 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:10.551122 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:10.551662 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:10.551676 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.553141 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.553146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:10.554389 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:10.554402 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.555527 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.555536 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:10.555706 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.556555 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:10.556575 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:10.557868 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:10.562438 (+ 4570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:10.562444 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:10.562448 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:10.562988 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:10.563710 (+ 722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:10.563718 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:10.563724 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:10.563932 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:10.564845 (+ 913us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:10.564853 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:10.565110 (+ 257us) client_negotiation.cc:769] Sending connection context
0929 15:51:10.565312 (+ 202us) client_negotiation.cc:240] Negotiation successful
0929 15:51:10.565754 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":479,"threads_started":1}
I20260929 15:51:10.567205 17270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:38111
I20260929 15:51:10.567804 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.548949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:35739 (local address 127.15.196.254:35441)
0929 15:51:10.549170 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:10.549177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:10.550053 (+ 876us) server_negotiation.cc:407] Connection header received
0929 15:51:10.550298 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:10.550304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:10.550384 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:10.550491 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:10.551834 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.552963 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.554651 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.555200 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.555387 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.558084 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:10.558090 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:10.558093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:10.558137 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:10.562286 (+ 4149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:10.563364 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:10.563370 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:10.563374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:10.563563 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:10.564110 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:10.564117 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:10.564123 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:10.564580 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:10.564758 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:10.566851 (+ 2093us) server_negotiation.cc:299] Negotiation successful
0929 15:51:10.567168 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 15:51:10.570853 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 17620
I20260929 15:51:10.571087 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:51:10.571718 16147 external_mini_cluster.cc:1428] Setting key 0e1d27cd0e888328ab223f3b230d09fb
I20260929 15:51:10.574756 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 15:51:10 dist-test-slave-c51v krb5kdc[17225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 15:51:10 dist-test-slave-c51v krb5kdc[17225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:10.620281 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.606440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60658 (local address 127.15.196.254:35441)
0929 15:51:10.606674 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:10.606681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:10.606929 (+ 248us) server_negotiation.cc:407] Connection header received
0929 15:51:10.607156 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:10.607161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:10.607226 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:10.607355 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:10.608712 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.609446 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.610719 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.611170 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.611376 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.615127 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:10.615132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:10.615135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:10.615169 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:10.617610 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:10.618244 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:10.618256 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:10.618259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:10.618384 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:10.618754 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:10.618760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:10.618763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:10.619071 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:10.619358 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:10.619677 (+ 319us) server_negotiation.cc:299] Negotiation successful
0929 15:51:10.619916 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 15:51:10.624377 17270 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:51:10.631095 17270 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:51:10.657404 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.643676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39462 (local address 127.15.196.195:41927)
0929 15:51:10.646155 (+ 2479us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:10.646166 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:10.646209 (+ 43us) server_negotiation.cc:407] Connection header received
0929 15:51:10.646339 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:10.646350 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:10.646639 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:10.646877 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:10.648190 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.650370 (+ 2180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.654019 (+ 3649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.656069 (+ 2050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.656279 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.656499 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:10.656873 (+ 374us) server_negotiation.cc:299] Negotiation successful
0929 15:51:10.657046 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2248,"thread_start_us":2063,"threads_started":1}
I20260929 15:51:10.657507 17769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.643429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:41927 (local address 127.0.0.1:39462)
0929 15:51:10.644411 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:10.644470 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:10.644729 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:10.647303 (+ 2574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:10.647324 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:10.647365 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:10.647894 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:10.647910 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.650568 (+ 2658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.650577 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:10.653773 (+ 3196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:10.653796 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.656374 (+ 2578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.656381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:10.656620 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.656685 (+ 65us) client_negotiation.cc:769] Sending connection context
0929 15:51:10.656821 (+ 136us) client_negotiation.cc:240] Negotiation successful
0929 15:51:10.657012 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"threads_started":1}
I20260929 15:51:10.662583 17685 tablet_service.cc:1467] Processing CreateTablet for tablet 83438693c67f4ed2a72730b46b0f1bf6 (DEFAULT_TABLE table=test-table [id=ac9a44e0c7964fb994074f8e6764d78e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:10.663911 17771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.643895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37872 (local address 127.15.196.193:35757)
0929 15:51:10.645557 (+ 1662us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:10.645570 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:10.656824 (+ 11254us) server_negotiation.cc:407] Connection header received
0929 15:51:10.656951 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:10.656963 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:10.657260 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:10.657490 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:10.658425 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.659582 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.661681 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.662786 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.662973 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.663156 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:10.663434 (+ 278us) server_negotiation.cc:299] Negotiation successful
0929 15:51:10.663604 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1503,"thread_start_us":1377,"threads_started":1}
I20260929 15:51:10.664165 17685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83438693c67f4ed2a72730b46b0f1bf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:10.664677 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.643540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:35757 (local address 127.0.0.1:37872)
0929 15:51:10.656605 (+ 13065us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:10.656621 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:10.656777 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:10.657855 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:10.657863 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:10.657892 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:10.658228 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:10.658240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.659768 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.659773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:10.661528 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:10.661540 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.663003 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.663009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:10.663173 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.663213 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 15:51:10.664114 (+ 901us) client_negotiation.cc:240] Negotiation successful
0929 15:51:10.664273 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12919,"mutex_wait_us":48,"thread_start_us":339,"threads_started":1}
I20260929 15:51:10.665282 17768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.645341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:34397 (local address 127.0.0.1:43812)
0929 15:51:10.657307 (+ 11966us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:10.657326 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:10.657464 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:10.658227 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:10.658232 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:10.658258 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:10.658654 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:10.658665 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.660197 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.660201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:10.662303 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:10.662317 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.664488 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.664493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:10.664689 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.664711 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 15:51:10.664838 (+ 127us) client_negotiation.cc:240] Negotiation successful
0929 15:51:10.664977 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11840,"thread_start_us":10879,"threads_started":1}
I20260929 15:51:10.665352 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.645559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43812 (local address 127.15.196.194:34397)
0929 15:51:10.656142 (+ 10583us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:10.656153 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:10.657689 (+ 1536us) server_negotiation.cc:407] Connection header received
0929 15:51:10.657814 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:10.657820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:10.658040 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:10.658240 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:10.658817 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.660018 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.662471 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.663950 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.664430 (+ 480us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.664601 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:10.664910 (+ 309us) server_negotiation.cc:299] Negotiation successful
0929 15:51:10.665068 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10403,"thread_start_us":10272,"threads_started":1}
I20260929 15:51:10.668766 17407 tablet_service.cc:1467] Processing CreateTablet for tablet 83438693c67f4ed2a72730b46b0f1bf6 (DEFAULT_TABLE table=test-table [id=ac9a44e0c7964fb994074f8e6764d78e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:10.668766 17546 tablet_service.cc:1467] Processing CreateTablet for tablet 83438693c67f4ed2a72730b46b0f1bf6 (DEFAULT_TABLE table=test-table [id=ac9a44e0c7964fb994074f8e6764d78e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:10.670176 17546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83438693c67f4ed2a72730b46b0f1bf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:10.670487 17407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83438693c67f4ed2a72730b46b0f1bf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:10.680066 17774 tablet_bootstrap.cc:492] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335: Bootstrap starting.
I20260929 15:51:10.683986 17774 tablet_bootstrap.cc:654] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:10.685037 17774 log.cc:824] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:10.685974 17775 tablet_bootstrap.cc:492] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0: Bootstrap starting.
I20260929 15:51:10.688041 17776 tablet_bootstrap.cc:492] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e: Bootstrap starting.
I20260929 15:51:10.689941 17774 tablet_bootstrap.cc:492] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335: No bootstrap required, opened a new log
I20260929 15:51:10.690218 17774 ts_tablet_manager.cc:1397] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260929 15:51:10.690853 17775 tablet_bootstrap.cc:654] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:10.691562 17776 tablet_bootstrap.cc:654] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:10.692255 17775 log.cc:824] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:10.692859 17776 log.cc:824] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:10.695143 17775 tablet_bootstrap.cc:492] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0: No bootstrap required, opened a new log
I20260929 15:51:10.695461 17775 ts_tablet_manager.cc:1397] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260929 15:51:10.695796 17776 tablet_bootstrap.cc:492] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e: No bootstrap required, opened a new log
I20260929 15:51:10.696059 17776 ts_tablet_manager.cc:1397] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260929 15:51:10.700157 17774 raft_consensus.cc:348] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } }
I20260929 15:51:10.700632 17774 raft_consensus.cc:374] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:10.700726 17774 raft_consensus.cc:729] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8fcc190371414fbc3f9dc19f539335, State: Initialized, Role: FOLLOWER
I20260929 15:51:10.701372 17774 consensus_queue.cc:260] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } }
I20260929 15:51:10.702502 17755 heartbeater.cc:503] Master 127.15.196.254:35441 was elected leader, sending a full tablet report...
I20260929 15:51:10.703181 17774 ts_tablet_manager.cc:1428] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335: Time spent starting tablet: real 0.013s user 0.005s sys 0.009s
I20260929 15:51:10.703933 17775 raft_consensus.cc:348] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } }
I20260929 15:51:10.704442 17775 raft_consensus.cc:374] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:10.704540 17775 raft_consensus.cc:729] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 138c5c0fd7024a288fc85c1d52cf71d0, State: Initialized, Role: FOLLOWER
I20260929 15:51:10.705137 17775 consensus_queue.cc:260] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } }
I20260929 15:51:10.706230 17616 heartbeater.cc:503] Master 127.15.196.254:35441 was elected leader, sending a full tablet report...
I20260929 15:51:10.706298 17776 raft_consensus.cc:348] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } }
I20260929 15:51:10.706696 17776 raft_consensus.cc:374] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:10.706825 17776 raft_consensus.cc:729] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 785a4dfd9aa34d3289e50ab9647ecf8e, State: Initialized, Role: FOLLOWER
I20260929 15:51:10.707453 17776 consensus_queue.cc:260] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } }
I20260929 15:51:10.708169 17775 ts_tablet_manager.cc:1428] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0: Time spent starting tablet: real 0.013s user 0.010s sys 0.000s
I20260929 15:51:10.710232 17776 ts_tablet_manager.cc:1428] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260929 15:51:10.754658 17780 raft_consensus.cc:482] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:51:10.755025 17780 raft_consensus.cc:504] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } }
I20260929 15:51:10.756788 17780 leader_election.cc:288] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 785a4dfd9aa34d3289e50ab9647ecf8e (127.15.196.193:35757), 138c5c0fd7024a288fc85c1d52cf71d0 (127.15.196.194:34397)
I20260929 15:51:10.762717 17644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.757052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:34397 (local address 127.15.196.195:57137)
0929 15:51:10.757444 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:10.757466 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:10.757597 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:10.757994 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:10.757999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:10.758025 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:10.758214 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:10.758221 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.759555 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.759560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:10.760889 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:10.760900 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.761964 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.761970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:10.762154 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.762178 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 15:51:10.762283 (+ 105us) client_negotiation.cc:240] Negotiation successful
0929 15:51:10.762428 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223}
I20260929 15:51:10.762698 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.757384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:57137 (local address 127.15.196.194:34397)
0929 15:51:10.757582 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:10.757588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:10.757609 (+ 21us) server_negotiation.cc:407] Connection header received
0929 15:51:10.757793 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:10.757799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:10.757871 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:10.757972 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:10.758349 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.759433 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.761044 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.761804 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.761982 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.762059 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:10.762308 (+ 249us) server_negotiation.cc:299] Negotiation successful
0929 15:51:10.762438 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 15:51:10.763384 17771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.757212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:50887 (local address 127.15.196.193:35757)
0929 15:51:10.757445 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:10.757453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:10.758086 (+ 633us) server_negotiation.cc:407] Connection header received
0929 15:51:10.758313 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:10.758319 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:10.758394 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:10.758486 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:10.759053 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.759958 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.761635 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.762397 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.762583 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.762658 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:10.763022 (+ 364us) server_negotiation.cc:299] Negotiation successful
0929 15:51:10.763158 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 15:51:10.763935 17783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.757197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:35757 (local address 127.15.196.195:50887)
0929 15:51:10.757978 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:10.757992 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:10.758148 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:10.758515 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:10.758519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:10.758543 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:10.758893 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:10.758904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.760093 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.760100 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:10.761471 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:10.761483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.762611 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.762617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:10.762813 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.762858 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 15:51:10.763573 (+ 715us) client_negotiation.cc:240] Negotiation successful
0929 15:51:10.763701 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":432,"threads_started":1}
I20260929 15:51:10.763798 17571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83438693c67f4ed2a72730b46b0f1bf6" candidate_uuid: "8c8fcc190371414fbc3f9dc19f539335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" is_pre_election: true
I20260929 15:51:10.764357 17571 raft_consensus.cc:2454] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c8fcc190371414fbc3f9dc19f539335 in term 0.
I20260929 15:51:10.764993 17432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83438693c67f4ed2a72730b46b0f1bf6" candidate_uuid: "8c8fcc190371414fbc3f9dc19f539335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" is_pre_election: true
I20260929 15:51:10.765081 17636 leader_election.cc:302] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 138c5c0fd7024a288fc85c1d52cf71d0, 8c8fcc190371414fbc3f9dc19f539335; no voters:
I20260929 15:51:10.765482 17780 raft_consensus.cc:2781] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:51:10.765451 17432 raft_consensus.cc:2454] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c8fcc190371414fbc3f9dc19f539335 in term 0.
I20260929 15:51:10.765592 17780 raft_consensus.cc:482] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:51:10.765703 17780 raft_consensus.cc:3037] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:10.768185 17780 raft_consensus.cc:504] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } }
I20260929 15:51:10.768744 17780 leader_election.cc:288] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [CANDIDATE]: Term 1 election: Requested vote from peers 785a4dfd9aa34d3289e50ab9647ecf8e (127.15.196.193:35757), 138c5c0fd7024a288fc85c1d52cf71d0 (127.15.196.194:34397)
I20260929 15:51:10.769137 17432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83438693c67f4ed2a72730b46b0f1bf6" candidate_uuid: "8c8fcc190371414fbc3f9dc19f539335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e"
I20260929 15:51:10.769235 17571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83438693c67f4ed2a72730b46b0f1bf6" candidate_uuid: "8c8fcc190371414fbc3f9dc19f539335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "138c5c0fd7024a288fc85c1d52cf71d0"
I20260929 15:51:10.769339 17432 raft_consensus.cc:3037] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:10.769474 17571 raft_consensus.cc:3037] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:10.771687 17432 raft_consensus.cc:2454] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c8fcc190371414fbc3f9dc19f539335 in term 1.
I20260929 15:51:10.771723 17571 raft_consensus.cc:2454] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c8fcc190371414fbc3f9dc19f539335 in term 1.
I20260929 15:51:10.772115 17636 leader_election.cc:302] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 138c5c0fd7024a288fc85c1d52cf71d0, 8c8fcc190371414fbc3f9dc19f539335; no voters:
I20260929 15:51:10.772428 17780 raft_consensus.cc:2781] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:10.772888 17780 raft_consensus.cc:686] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [term 1 LEADER]: Becoming Leader. State: Replica: 8c8fcc190371414fbc3f9dc19f539335, State: Running, Role: LEADER
I20260929 15:51:10.773360 17780 consensus_queue.cc:237] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } }
W20260929 15:51:10.774880 17478 tablet.cc:2369] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:51:10.778384 17270 catalog_manager.cc:5495] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c8fcc190371414fbc3f9dc19f539335 (127.15.196.195). New cstate: current_term: 1 leader_uuid: "8c8fcc190371414fbc3f9dc19f539335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } health_report { overall_health: UNKNOWN } } }
W20260929 15:51:10.792781 17617 tablet.cc:2369] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:51:10.800264 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:10.795230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39474 (local address 127.15.196.195:41927)
0929 15:51:10.795471 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:10.795476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:10.795495 (+ 19us) server_negotiation.cc:407] Connection header received
0929 15:51:10.795725 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:10.795729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:10.795803 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:10.795917 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:51:10.796221 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.796906 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.798177 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:10.798585 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:10.798677 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:10.799107 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:51:10.799673 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:51:10.799784 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:10.799898 (+ 114us) server_negotiation.cc:299] Negotiation successful
0929 15:51:10.799963 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260929 15:51:10.808727 17432 raft_consensus.cc:1261] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e [term 1 FOLLOWER]: Refusing update from remote peer 8c8fcc190371414fbc3f9dc19f539335: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:10.808723 17571 raft_consensus.cc:1261] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0 [term 1 FOLLOWER]: Refusing update from remote peer 8c8fcc190371414fbc3f9dc19f539335: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:10.809823 17784 consensus_queue.cc:1035] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:10.810170 17780 consensus_queue.cc:1035] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 15:51:10.818779 17756 tablet.cc:2369] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:51:10.821420 17787 mvcc.cc:205] Tried to move back new op lower bound from 7334695202021670912 to 7334695201891647488. Current Snapshot: MvccSnapshot[applied={T|T < 7334695202021670912}]
I20260929 15:51:10.839582 17270 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:10.839854 17270 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:10.843604 17270 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed: Sending DeleteTablet for 3 replicas of tablet 83438693c67f4ed2a72730b46b0f1bf6
I20260929 15:51:10.845086 17407 tablet_service.cc:1514] Processing DeleteTablet for tablet 83438693c67f4ed2a72730b46b0f1bf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:10 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:37872
I20260929 15:51:10.845088 17546 tablet_service.cc:1514] Processing DeleteTablet for tablet 83438693c67f4ed2a72730b46b0f1bf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:10 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:43812
I20260929 15:51:10.845120 17685 tablet_service.cc:1514] Processing DeleteTablet for tablet 83438693c67f4ed2a72730b46b0f1bf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:10 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:39462
I20260929 15:51:10.846146 17796 tablet_replica.cc:330] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335: stopping tablet replica
I20260929 15:51:10.846698 17796 raft_consensus.cc:2229] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:51:10.847551 17796 raft_consensus.cc:2258] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:10.848122 17268 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:51:10.849057 17268 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:51:10.850030 17796 ts_tablet_manager.cc:1905] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:10.857378 17798 tablet_replica.cc:330] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0: stopping tablet replica
I20260929 15:51:10.858029 17796 ts_tablet_manager.cc:1918] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:51:10.858268 17796 log.cc:1196] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/83438693c67f4ed2a72730b46b0f1bf6
I20260929 15:51:10.858234 17798 raft_consensus.cc:2229] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:10.858392 17685 tablet_service.cc:1467] Processing CreateTablet for tablet 83d485163af54a95b4941339486b430e (DEFAULT_TABLE table=test-table [id=3fb91807b5bc40f8b6a91b47f7a676a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:10.858388 17406 tablet_service.cc:1467] Processing CreateTablet for tablet 83d485163af54a95b4941339486b430e (DEFAULT_TABLE table=test-table [id=3fb91807b5bc40f8b6a91b47f7a676a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:10.858791 17798 raft_consensus.cc:2258] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:10.858830 17685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83d485163af54a95b4941339486b430e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:10.858968 17406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83d485163af54a95b4941339486b430e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:10.859185 17796 ts_tablet_manager.cc:1939] T 83438693c67f4ed2a72730b46b0f1bf6 P 8c8fcc190371414fbc3f9dc19f539335: Deleting consensus metadata
I20260929 15:51:10.860955 17256 catalog_manager.cc:4841] TS 8c8fcc190371414fbc3f9dc19f539335 (127.15.196.195:41927): tablet 83438693c67f4ed2a72730b46b0f1bf6 (table test-table [id=ac9a44e0c7964fb994074f8e6764d78e]) successfully deleted
I20260929 15:51:10.862811 17798 ts_tablet_manager.cc:1905] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:10.864044 17546 tablet_service.cc:1467] Processing CreateTablet for tablet 83d485163af54a95b4941339486b430e (DEFAULT_TABLE table=test-table [id=3fb91807b5bc40f8b6a91b47f7a676a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:10.864589 17546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83d485163af54a95b4941339486b430e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:10.864629 17776 tablet_bootstrap.cc:492] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e: Bootstrap starting.
I20260929 15:51:10.865345 17774 tablet_bootstrap.cc:492] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335: Bootstrap starting.
I20260929 15:51:10.866300 17776 tablet_bootstrap.cc:654] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:10.866899 17774 tablet_bootstrap.cc:654] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:10.871724 17775 tablet_bootstrap.cc:492] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0: Bootstrap starting.
I20260929 15:51:10.874046 17775 tablet_bootstrap.cc:654] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:10.875746 17774 tablet_bootstrap.cc:492] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335: No bootstrap required, opened a new log
I20260929 15:51:10.875901 17774 ts_tablet_manager.cc:1397] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260929 15:51:10.876430 17775 tablet_bootstrap.cc:492] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0: No bootstrap required, opened a new log
I20260929 15:51:10.876569 17775 ts_tablet_manager.cc:1397] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260929 15:51:10.876756 17774 raft_consensus.cc:348] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:10.876972 17774 raft_consensus.cc:374] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:10.877056 17774 raft_consensus.cc:729] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8fcc190371414fbc3f9dc19f539335, State: Initialized, Role: FOLLOWER
I20260929 15:51:10.877333 17774 consensus_queue.cc:260] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:10.877450 17775 raft_consensus.cc:348] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:10.877648 17775 raft_consensus.cc:374] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:10.877738 17775 raft_consensus.cc:729] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 138c5c0fd7024a288fc85c1d52cf71d0, State: Initialized, Role: FOLLOWER
I20260929 15:51:10.877923 17774 ts_tablet_manager.cc:1428] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 15:51:10.877992 17775 consensus_queue.cc:260] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:10.878525 17775 ts_tablet_manager.cc:1428] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 15:51:10.883556 17797 tablet_replica.cc:330] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e: stopping tablet replica
I20260929 15:51:10.884001 17797 raft_consensus.cc:2229] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:10.884491 17797 raft_consensus.cc:2258] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:10.886238 17797 ts_tablet_manager.cc:1905] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:10.887848 17798 ts_tablet_manager.cc:1918] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:51:10.888091 17798 log.cc:1196] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/83438693c67f4ed2a72730b46b0f1bf6
I20260929 15:51:10.888798 17798 ts_tablet_manager.cc:1939] T 83438693c67f4ed2a72730b46b0f1bf6 P 138c5c0fd7024a288fc85c1d52cf71d0: Deleting consensus metadata
I20260929 15:51:10.890620 17254 catalog_manager.cc:4841] TS 138c5c0fd7024a288fc85c1d52cf71d0 (127.15.196.194:34397): tablet 83438693c67f4ed2a72730b46b0f1bf6 (table test-table [id=ac9a44e0c7964fb994074f8e6764d78e]) successfully deleted
I20260929 15:51:10.891747 17776 tablet_bootstrap.cc:492] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e: No bootstrap required, opened a new log
I20260929 15:51:10.891940 17776 ts_tablet_manager.cc:1397] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e: Time spent bootstrapping tablet: real 0.027s user 0.004s sys 0.008s
I20260929 15:51:10.892940 17776 raft_consensus.cc:348] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:10.893208 17776 raft_consensus.cc:374] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:10.893314 17776 raft_consensus.cc:729] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 785a4dfd9aa34d3289e50ab9647ecf8e, State: Initialized, Role: FOLLOWER
I20260929 15:51:10.893599 17776 consensus_queue.cc:260] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:10.895020 17797 ts_tablet_manager.cc:1918] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:51:10.895119 17776 ts_tablet_manager.cc:1428] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260929 15:51:10.895188 17797 log.cc:1196] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/83438693c67f4ed2a72730b46b0f1bf6
I20260929 15:51:10.895802 17797 ts_tablet_manager.cc:1939] T 83438693c67f4ed2a72730b46b0f1bf6 P 785a4dfd9aa34d3289e50ab9647ecf8e: Deleting consensus metadata
I20260929 15:51:10.897800 17255 catalog_manager.cc:4841] TS 785a4dfd9aa34d3289e50ab9647ecf8e (127.15.196.193:35757): tablet 83438693c67f4ed2a72730b46b0f1bf6 (table test-table [id=ac9a44e0c7964fb994074f8e6764d78e]) successfully deleted
I20260929 15:51:11.130131 17780 raft_consensus.cc:482] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:51:11.130403 17780 raft_consensus.cc:504] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:11.131102 17780 leader_election.cc:288] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 785a4dfd9aa34d3289e50ab9647ecf8e (127.15.196.193:35757), 138c5c0fd7024a288fc85c1d52cf71d0 (127.15.196.194:34397)
I20260929 15:51:11.131752 17571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83d485163af54a95b4941339486b430e" candidate_uuid: "8c8fcc190371414fbc3f9dc19f539335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" is_pre_election: true
I20260929 15:51:11.131831 17432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83d485163af54a95b4941339486b430e" candidate_uuid: "8c8fcc190371414fbc3f9dc19f539335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" is_pre_election: true
I20260929 15:51:11.132017 17571 raft_consensus.cc:2454] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c8fcc190371414fbc3f9dc19f539335 in term 0.
I20260929 15:51:11.132140 17432 raft_consensus.cc:2454] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c8fcc190371414fbc3f9dc19f539335 in term 0.
I20260929 15:51:11.132550 17636 leader_election.cc:302] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 138c5c0fd7024a288fc85c1d52cf71d0, 8c8fcc190371414fbc3f9dc19f539335; no voters:
I20260929 15:51:11.132809 17780 raft_consensus.cc:2781] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:51:11.132930 17780 raft_consensus.cc:482] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:51:11.133076 17780 raft_consensus.cc:3037] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:11.135252 17780 raft_consensus.cc:504] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:11.136042 17780 leader_election.cc:288] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [CANDIDATE]: Term 1 election: Requested vote from peers 785a4dfd9aa34d3289e50ab9647ecf8e (127.15.196.193:35757), 138c5c0fd7024a288fc85c1d52cf71d0 (127.15.196.194:34397)
I20260929 15:51:11.136554 17571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83d485163af54a95b4941339486b430e" candidate_uuid: "8c8fcc190371414fbc3f9dc19f539335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "138c5c0fd7024a288fc85c1d52cf71d0"
I20260929 15:51:11.136516 17432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83d485163af54a95b4941339486b430e" candidate_uuid: "8c8fcc190371414fbc3f9dc19f539335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e"
I20260929 15:51:11.136770 17571 raft_consensus.cc:3037] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:11.136771 17432 raft_consensus.cc:3037] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:11.138772 17571 raft_consensus.cc:2454] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c8fcc190371414fbc3f9dc19f539335 in term 1.
I20260929 15:51:11.139240 17636 leader_election.cc:302] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 138c5c0fd7024a288fc85c1d52cf71d0, 8c8fcc190371414fbc3f9dc19f539335; no voters:
I20260929 15:51:11.139559 17780 raft_consensus.cc:2781] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:11.139859 17432 raft_consensus.cc:2454] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c8fcc190371414fbc3f9dc19f539335 in term 1.
I20260929 15:51:11.139729 17780 raft_consensus.cc:686] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [term 1 LEADER]: Becoming Leader. State: Replica: 8c8fcc190371414fbc3f9dc19f539335, State: Running, Role: LEADER
I20260929 15:51:11.140348 17780 consensus_queue.cc:237] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:11.143126 17268 catalog_manager.cc:5495] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c8fcc190371414fbc3f9dc19f539335 (127.15.196.195). New cstate: current_term: 1 leader_uuid: "8c8fcc190371414fbc3f9dc19f539335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } health_report { overall_health: HEALTHY } } }
Sep 29 15:51:11 dist-test-slave-c51v krb5kdc[17225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697068, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:11.201327 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:11.185725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:35441 (local address 127.0.0.1:60664)
0929 15:51:11.186441 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:11.186463 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:11.186631 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:11.187075 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:11.187081 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:11.188035 (+ 954us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:11.188320 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:11.188334 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.189409 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.189416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:11.191017 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:11.191032 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.191802 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.191810 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:11.191999 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:11.192766 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:11.192793 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:11.195425 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:11.198431 (+ 3006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:11.198442 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:11.198450 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:11.198849 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:11.199210 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:11.199217 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:11.199222 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:11.199430 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:11.200182 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:11.200189 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:11.200355 (+ 166us) client_negotiation.cc:769] Sending connection context
0929 15:51:11.200525 (+ 170us) client_negotiation.cc:240] Negotiation successful
0929 15:51:11.200843 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"spinlock_wait_cycles":1408,"thread_start_us":356,"threads_started":1}
I20260929 15:51:11.201366 17808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:11.185741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60664 (local address 127.15.196.254:35441)
0929 15:51:11.186782 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:11.186787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:11.186807 (+ 20us) server_negotiation.cc:407] Connection header received
0929 15:51:11.186852 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:11.186855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:11.186907 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:11.186996 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:11.188490 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.189248 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.191163 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.191611 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.191705 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:11.195608 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:11.195613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:11.195616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:11.195643 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:11.198281 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:11.198962 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:11.198969 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:11.198972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:11.199100 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:11.199713 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:11.199721 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:11.199725 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:11.200024 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:11.200197 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:11.200626 (+ 429us) server_negotiation.cc:299] Negotiation successful
0929 15:51:11.200966 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":151,"server-negotiator.queue_time_us":950,"thread_start_us":826,"threads_started":1}
I20260929 15:51:11.206950 17268 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:60664:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 15:51:11.207994 17268 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:51:11.215013 17407 tablet_service.cc:1467] Processing CreateTablet for tablet a63aca7a14cf45c89473b780cef511e3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6165cd1d5514e08bd0768fe6536fcc4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 15:51:11.215119 17546 tablet_service.cc:1467] Processing CreateTablet for tablet a63aca7a14cf45c89473b780cef511e3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6165cd1d5514e08bd0768fe6536fcc4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 15:51:11.215636 17407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a63aca7a14cf45c89473b780cef511e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:11.215641 17546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a63aca7a14cf45c89473b780cef511e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:11.215571 17685 tablet_service.cc:1467] Processing CreateTablet for tablet a63aca7a14cf45c89473b780cef511e3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6165cd1d5514e08bd0768fe6536fcc4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 15:51:11.216195 17685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a63aca7a14cf45c89473b780cef511e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:11.223044 17775 tablet_bootstrap.cc:492] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0: Bootstrap starting.
I20260929 15:51:11.223071 17776 tablet_bootstrap.cc:492] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e: Bootstrap starting.
I20260929 15:51:11.224436 17775 tablet_bootstrap.cc:654] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:11.225060 17776 tablet_bootstrap.cc:654] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:11.226584 17775 tablet_bootstrap.cc:492] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0: No bootstrap required, opened a new log
I20260929 15:51:11.226774 17775 ts_tablet_manager.cc:1397] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260929 15:51:11.227495 17775 raft_consensus.cc:348] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:11.227773 17775 raft_consensus.cc:374] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:11.227862 17775 raft_consensus.cc:729] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 138c5c0fd7024a288fc85c1d52cf71d0, State: Initialized, Role: FOLLOWER
I20260929 15:51:11.228111 17775 consensus_queue.cc:260] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:11.228754 17774 tablet_bootstrap.cc:492] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335: Bootstrap starting.
I20260929 15:51:11.229183 17781 tablet_replica.cc:439] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } }
I20260929 15:51:11.229499 17781 tablet_replica.cc:442] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:51:11.230132 17776 tablet_bootstrap.cc:492] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e: No bootstrap required, opened a new log
I20260929 15:51:11.230324 17776 ts_tablet_manager.cc:1397] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260929 15:51:11.230618 17774 tablet_bootstrap.cc:654] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:11.231137 17776 raft_consensus.cc:348] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:11.231436 17776 raft_consensus.cc:374] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:11.231521 17776 raft_consensus.cc:729] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 785a4dfd9aa34d3289e50ab9647ecf8e, State: Initialized, Role: FOLLOWER
I20260929 15:51:11.231868 17776 consensus_queue.cc:260] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:11.232448 17775 ts_tablet_manager.cc:1428] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260929 15:51:11.232398 17782 tablet_replica.cc:439] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } }
I20260929 15:51:11.232834 17782 tablet_replica.cc:442] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:51:11.233349 17776 ts_tablet_manager.cc:1428] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:11.238296 17774 tablet_bootstrap.cc:492] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335: No bootstrap required, opened a new log
I20260929 15:51:11.238492 17774 ts_tablet_manager.cc:1397] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260929 15:51:11.239208 17774 raft_consensus.cc:348] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:11.239507 17774 raft_consensus.cc:374] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:11.239595 17774 raft_consensus.cc:729] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8fcc190371414fbc3f9dc19f539335, State: Initialized, Role: FOLLOWER
I20260929 15:51:11.239858 17774 consensus_queue.cc:260] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:11.240443 17774 ts_tablet_manager.cc:1428] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 15:51:11.240382 17780 tablet_replica.cc:439] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } }
I20260929 15:51:11.240837 17780 tablet_replica.cc:442] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:51:11.264161 17780 raft_consensus.cc:482] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:51:11.264416 17780 raft_consensus.cc:504] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:11.265101 17780 leader_election.cc:288] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 785a4dfd9aa34d3289e50ab9647ecf8e (127.15.196.193:35757), 138c5c0fd7024a288fc85c1d52cf71d0 (127.15.196.194:34397)
I20260929 15:51:11.265621 17571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a63aca7a14cf45c89473b780cef511e3" candidate_uuid: "8c8fcc190371414fbc3f9dc19f539335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" is_pre_election: true
I20260929 15:51:11.265658 17432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a63aca7a14cf45c89473b780cef511e3" candidate_uuid: "8c8fcc190371414fbc3f9dc19f539335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" is_pre_election: true
I20260929 15:51:11.265890 17571 raft_consensus.cc:2454] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c8fcc190371414fbc3f9dc19f539335 in term 0.
I20260929 15:51:11.265950 17432 raft_consensus.cc:2454] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c8fcc190371414fbc3f9dc19f539335 in term 0.
I20260929 15:51:11.266410 17636 leader_election.cc:302] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 138c5c0fd7024a288fc85c1d52cf71d0, 8c8fcc190371414fbc3f9dc19f539335; no voters:
I20260929 15:51:11.266737 17780 raft_consensus.cc:2781] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:51:11.266851 17780 raft_consensus.cc:482] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:51:11.266992 17780 raft_consensus.cc:3037] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:11.269404 17780 raft_consensus.cc:504] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:11.270041 17780 leader_election.cc:288] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [CANDIDATE]: Term 1 election: Requested vote from peers 785a4dfd9aa34d3289e50ab9647ecf8e (127.15.196.193:35757), 138c5c0fd7024a288fc85c1d52cf71d0 (127.15.196.194:34397)
I20260929 15:51:11.270543 17571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a63aca7a14cf45c89473b780cef511e3" candidate_uuid: "8c8fcc190371414fbc3f9dc19f539335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "138c5c0fd7024a288fc85c1d52cf71d0"
I20260929 15:51:11.270543 17432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a63aca7a14cf45c89473b780cef511e3" candidate_uuid: "8c8fcc190371414fbc3f9dc19f539335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e"
I20260929 15:51:11.270798 17571 raft_consensus.cc:3037] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:11.270787 17432 raft_consensus.cc:3037] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:11.273747 17432 raft_consensus.cc:2454] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c8fcc190371414fbc3f9dc19f539335 in term 1.
I20260929 15:51:11.273747 17571 raft_consensus.cc:2454] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c8fcc190371414fbc3f9dc19f539335 in term 1.
I20260929 15:51:11.274139 17636 leader_election.cc:302] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 138c5c0fd7024a288fc85c1d52cf71d0, 8c8fcc190371414fbc3f9dc19f539335; no voters:
I20260929 15:51:11.274422 17780 raft_consensus.cc:2781] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:11.274595 17780 raft_consensus.cc:686] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [term 1 LEADER]: Becoming Leader. State: Replica: 8c8fcc190371414fbc3f9dc19f539335, State: Running, Role: LEADER
I20260929 15:51:11.274890 17780 consensus_queue.cc:237] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } }
I20260929 15:51:11.276705 17784 tablet_replica.cc:439] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335: TxnStatusTablet state changed. Reason: New leader 8c8fcc190371414fbc3f9dc19f539335. Latest consensus state: current_term: 1 leader_uuid: "8c8fcc190371414fbc3f9dc19f539335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } }
I20260929 15:51:11.277035 17784 tablet_replica.cc:442] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335: This TxnStatusTablet replica's current role is: LEADER
I20260929 15:51:11.277971 17814 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 15:51:11.278230 17814 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 15:51:11.278416 17270 catalog_manager.cc:5495] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c8fcc190371414fbc3f9dc19f539335 (127.15.196.195). New cstate: current_term: 1 leader_uuid: "8c8fcc190371414fbc3f9dc19f539335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } health_report { overall_health: HEALTHY } } }
I20260929 15:51:11.299782 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:11.293152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39484 (local address 127.15.196.195:41927)
0929 15:51:11.293350 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:11.293358 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:11.293493 (+ 135us) server_negotiation.cc:407] Connection header received
0929 15:51:11.293757 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:11.293761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:11.293890 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:11.293981 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:51:11.294397 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.295456 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.297601 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.298034 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.298151 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:11.298927 (+ 776us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:51:11.299096 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:51:11.299189 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:11.299396 (+ 207us) server_negotiation.cc:299] Negotiation successful
0929 15:51:11.299470 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 15:51:11.299887 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:11.293173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:41927 (local address 127.0.0.1:39484)
0929 15:51:11.293395 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:11.293419 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:11.293571 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:11.294031 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:11.294037 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:11.294051 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 15:51:11.294260 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:11.294271 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.295590 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.295597 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:11.297440 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:11.297458 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.298198 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.298204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:11.298483 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:11.298582 (+ 99us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 15:51:11.299215 (+ 633us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 15:51:11.299235 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 15:51:11.299359 (+ 124us) client_negotiation.cc:240] Negotiation successful
0929 15:51:11.299531 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 15:51:11.304054 17571 raft_consensus.cc:1261] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0 [term 1 FOLLOWER]: Refusing update from remote peer 8c8fcc190371414fbc3f9dc19f539335: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:11.304216 17432 raft_consensus.cc:1261] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e [term 1 FOLLOWER]: Refusing update from remote peer 8c8fcc190371414fbc3f9dc19f539335: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:11.304728 17780 consensus_queue.cc:1035] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:11.304999 17784 consensus_queue.cc:1035] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:11.308349 17782 tablet_replica.cc:439] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e: TxnStatusTablet state changed. Reason: New leader 8c8fcc190371414fbc3f9dc19f539335. Latest consensus state: current_term: 1 leader_uuid: "8c8fcc190371414fbc3f9dc19f539335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } }
I20260929 15:51:11.308602 17782 tablet_replica.cc:442] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:51:11.310321 17780 tablet_replica.cc:439] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c8fcc190371414fbc3f9dc19f539335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } }
I20260929 15:51:11.311427 17780 tablet_replica.cc:442] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335: This TxnStatusTablet replica's current role is: LEADER
I20260929 15:51:11.313367 17789 mvcc.cc:205] Tried to move back new op lower bound from 7334695204054249472 to 7334695203944525824. Current Snapshot: MvccSnapshot[applied={T|T < 7334695204054249472}]
I20260929 15:51:11.313959 17782 tablet_replica.cc:439] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c8fcc190371414fbc3f9dc19f539335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } }
I20260929 15:51:11.314215 17782 tablet_replica.cc:442] T a63aca7a14cf45c89473b780cef511e3 P 785a4dfd9aa34d3289e50ab9647ecf8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:51:11.321113 17781 tablet_replica.cc:439] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0: TxnStatusTablet state changed. Reason: New leader 8c8fcc190371414fbc3f9dc19f539335. Latest consensus state: current_term: 1 leader_uuid: "8c8fcc190371414fbc3f9dc19f539335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } }
I20260929 15:51:11.321412 17781 tablet_replica.cc:442] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:51:11.322594 17784 tablet_replica.cc:439] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c8fcc190371414fbc3f9dc19f539335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } }
I20260929 15:51:11.322868 17784 tablet_replica.cc:442] T a63aca7a14cf45c89473b780cef511e3 P 8c8fcc190371414fbc3f9dc19f539335: This TxnStatusTablet replica's current role is: LEADER
I20260929 15:51:11.324059 17781 tablet_replica.cc:439] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c8fcc190371414fbc3f9dc19f539335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 } } peers { permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 } } peers { permanent_uuid: "8c8fcc190371414fbc3f9dc19f539335" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 41927 } } }
I20260929 15:51:11.324311 17781 tablet_replica.cc:442] T a63aca7a14cf45c89473b780cef511e3 P 138c5c0fd7024a288fc85c1d52cf71d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:51:11.339942 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:11.333756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:42075 (local address 127.15.196.195:41927)
0929 15:51:11.333923 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:11.333928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:11.334601 (+ 673us) server_negotiation.cc:407] Connection header received
0929 15:51:11.334839 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:11.334844 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:11.334925 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:11.335025 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:51:11.335642 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.336423 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.337990 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.338485 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.338636 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:11.339112 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:51:11.339247 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:51:11.339388 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:11.339534 (+ 146us) server_negotiation.cc:299] Negotiation successful
0929 15:51:11.339603 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 15:51:11.340126 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:11.333829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:41927 (local address 127.15.196.195:42075)
0929 15:51:11.334490 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:11.334506 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:11.334697 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:11.335034 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:11.335039 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:11.335050 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 15:51:11.335477 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:11.335488 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.336579 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.336584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:11.337802 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:11.337814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.338676 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.338682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:11.338845 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:11.338898 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 15:51:11.339388 (+ 490us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 15:51:11.339401 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 15:51:11.339514 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 15:51:11.339653 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":332,"threads_started":1}
I20260929 15:51:11.351090 17432 raft_consensus.cc:1261] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e [term 1 FOLLOWER]: Refusing update from remote peer 8c8fcc190371414fbc3f9dc19f539335: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:11.351655 17780 consensus_queue.cc:1035] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "785a4dfd9aa34d3289e50ab9647ecf8e" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:11.351744 17571 raft_consensus.cc:1261] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0 [term 1 FOLLOWER]: Refusing update from remote peer 8c8fcc190371414fbc3f9dc19f539335: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:11.352301 17780 consensus_queue.cc:1035] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "138c5c0fd7024a288fc85c1d52cf71d0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 34397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:11.412140 17270 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:11.412355 17270 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:11.415573 17270 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 032855516d3642db8cde6bf44d13c1ed: Sending DeleteTablet for 3 replicas of tablet 83d485163af54a95b4941339486b430e
I20260929 15:51:11.416356 17407 tablet_service.cc:1514] Processing DeleteTablet for tablet 83d485163af54a95b4941339486b430e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:11 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:37872
I20260929 15:51:11.416419 17546 tablet_service.cc:1514] Processing DeleteTablet for tablet 83d485163af54a95b4941339486b430e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:11 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:43812
I20260929 15:51:11.416538 17684 tablet_service.cc:1514] Processing DeleteTablet for tablet 83d485163af54a95b4941339486b430e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:11 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:39462
I20260929 15:51:11.428807 17825 tablet_replica.cc:330] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e: stopping tablet replica
I20260929 15:51:11.428985 17826 tablet_replica.cc:330] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0: stopping tablet replica
I20260929 15:51:11.429167 17825 raft_consensus.cc:2229] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:11.429227 17826 raft_consensus.cc:2229] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:11.429472 17826 raft_consensus.cc:2258] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:11.429471 17825 raft_consensus.cc:2258] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:11.430622 17825 ts_tablet_manager.cc:1905] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:11.430622 17826 ts_tablet_manager.cc:1905] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:11.437904 17826 ts_tablet_manager.cc:1918] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 15:51:11.437904 17825 ts_tablet_manager.cc:1918] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 15:51:11.438042 17825 log.cc:1196] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/83d485163af54a95b4941339486b430e
I20260929 15:51:11.438133 17826 log.cc:1196] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/83d485163af54a95b4941339486b430e
I20260929 15:51:11.438594 17825 ts_tablet_manager.cc:1939] T 83d485163af54a95b4941339486b430e P 785a4dfd9aa34d3289e50ab9647ecf8e: Deleting consensus metadata
I20260929 15:51:11.438630 17826 ts_tablet_manager.cc:1939] T 83d485163af54a95b4941339486b430e P 138c5c0fd7024a288fc85c1d52cf71d0: Deleting consensus metadata
I20260929 15:51:11.439754 17255 catalog_manager.cc:4841] TS 785a4dfd9aa34d3289e50ab9647ecf8e (127.15.196.193:35757): tablet 83d485163af54a95b4941339486b430e (table test-table [id=3fb91807b5bc40f8b6a91b47f7a676a9]) successfully deleted
I20260929 15:51:11.440027 17254 catalog_manager.cc:4841] TS 138c5c0fd7024a288fc85c1d52cf71d0 (127.15.196.194:34397): tablet 83d485163af54a95b4941339486b430e (table test-table [id=3fb91807b5bc40f8b6a91b47f7a676a9]) successfully deleted
I20260929 15:51:11.441282 17827 tablet_replica.cc:330] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335: stopping tablet replica
I20260929 15:51:11.441555 17827 raft_consensus.cc:2229] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:51:11.441967 17827 raft_consensus.cc:2258] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:11.443075 17827 ts_tablet_manager.cc:1905] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:11.449738 17827 ts_tablet_manager.cc:1918] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 15:51:11.449903 17827 log.cc:1196] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/83d485163af54a95b4941339486b430e
I20260929 15:51:11.450332 17827 ts_tablet_manager.cc:1939] T 83d485163af54a95b4941339486b430e P 8c8fcc190371414fbc3f9dc19f539335: Deleting consensus metadata
I20260929 15:51:11.451457 17256 catalog_manager.cc:4841] TS 8c8fcc190371414fbc3f9dc19f539335 (127.15.196.195:41927): tablet 83d485163af54a95b4941339486b430e (table test-table [id=3fb91807b5bc40f8b6a91b47f7a676a9]) successfully deleted
Sep 29 15:51:11 dist-test-slave-c51v krb5kdc[17225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:51:11.463626 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:11.439683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37884 (local address 127.15.196.193:35757)
0929 15:51:11.440345 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:11.440351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:11.441112 (+ 761us) server_negotiation.cc:407] Connection header received
0929 15:51:11.441301 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:11.441309 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:11.441391 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:11.441507 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:11.443496 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.444735 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.446884 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.447522 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.447651 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:11.453185 (+ 5534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:11.453199 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:11.453215 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:11.453248 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:11.460615 (+ 7367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:11.461549 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:11.461555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:11.461558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:11.461706 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:11.462133 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:11.462138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:11.462140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:11.462484 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:11.462698 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:11.463086 (+ 388us) server_negotiation.cc:299] Negotiation successful
0929 15:51:11.463261 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":570,"threads_started":1}
W20260929 15:51:11.464370 17472 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37884
I20260929 15:51:11.480171 17808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:11.467259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60670 (local address 127.15.196.254:35441)
0929 15:51:11.467540 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:11.467548 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:11.467984 (+ 436us) server_negotiation.cc:407] Connection header received
0929 15:51:11.468200 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:11.468206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:11.468283 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:11.468422 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:11.469787 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.470654 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.472055 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.472478 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.472609 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:11.474856 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:11.474863 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:11.474867 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:11.474909 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:11.477562 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:11.478243 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:11.478249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:11.478253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:11.478391 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:11.478739 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:11.478745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:11.478748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:11.479017 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:11.479269 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:11.479638 (+ 369us) server_negotiation.cc:299] Negotiation successful
0929 15:51:11.479864 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
W20260929 15:51:11.480773 17270 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60670
I20260929 15:51:11.482553 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 17341
I20260929 15:51:11.497889 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:11.483801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37890 (local address 127.15.196.193:35757)
0929 15:51:11.483998 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:11.484002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:11.484113 (+ 111us) server_negotiation.cc:407] Connection header received
0929 15:51:11.484277 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:11.484281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:11.484348 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:11.484430 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:11.486127 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.486811 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.488712 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.489114 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.489207 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:11.492525 (+ 3318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:11.492530 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:11.492533 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:11.492566 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:11.495259 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:11.496164 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:11.496167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:11.496170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:11.496287 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:11.496669 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:11.496672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:11.496674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:11.496921 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:11.497098 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:11.497474 (+ 376us) server_negotiation.cc:299] Negotiation successful
0929 15:51:11.497641 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260929 15:51:11.498605 17472 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37890
W20260929 15:51:11.499191 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 17341: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:51:11.499408 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 17341
I20260929 15:51:11.517483 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 17481
Sep 29 15:51:11 dist-test-slave-c51v krb5kdc[17225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:51:11.536362 17834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:11.518335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43824 (local address 127.15.196.194:34397)
0929 15:51:11.519187 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:11.519191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:11.519208 (+ 17us) server_negotiation.cc:407] Connection header received
0929 15:51:11.519259 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:11.519264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:11.519401 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:11.519565 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:11.521156 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.521974 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.523933 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.524321 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.524421 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:11.529609 (+ 5188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:11.529625 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:11.529641 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:11.529674 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:11.533197 (+ 3523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:11.533988 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:11.533997 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:11.533999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:11.534140 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:11.534593 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:11.534601 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:11.534605 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:11.534993 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:11.535374 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:11.535686 (+ 312us) server_negotiation.cc:299] Negotiation successful
0929 15:51:11.535949 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":602,"threads_started":1}
W20260929 15:51:11.537228 17611 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43824
W20260929 15:51:11.538002 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 17481: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:51:11.538232 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 17481
I20260929 15:51:11.561640 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 17620
Sep 29 15:51:11 dist-test-slave-c51v krb5kdc[17225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697070, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:51:11.583751 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:11.562446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39496 (local address 127.15.196.195:41927)
0929 15:51:11.562636 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:11.562642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:11.562762 (+ 120us) server_negotiation.cc:407] Connection header received
0929 15:51:11.562989 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:11.562995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:11.563090 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:11.563211 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:11.564810 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.566043 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.568004 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:11.568869 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:11.569085 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:11.575958 (+ 6873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:11.575974 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:11.575994 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:11.576040 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:11.580232 (+ 4192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:11.581066 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:11.581078 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:11.581087 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:11.581240 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:11.581649 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:11.581654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:11.581658 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:11.582159 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:11.582495 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:11.582803 (+ 308us) server_negotiation.cc:299] Negotiation successful
0929 15:51:11.583029 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260929 15:51:11.584808 17750 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39496
W20260929 15:51:11.585667 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 17620: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:51:11.585826 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 17620
I20260929 15:51:11.608506 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 17238
I20260929 15:51:11.609625 17310 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:11.736001 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 17238
2026-09-29T15:51:11Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4558 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:51:11 dist-test-slave-c51v krb5kdc[17838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:51:11 dist-test-slave-c51v krb5kdc[17838](info): set up 2 sockets
Sep 29 15:51:11 dist-test-slave-c51v krb5kdc[17838](info): commencing operation
krb5kdc: starting...
W20260929 15:51:12.837666 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:51:12 dist-test-slave-c51v krb5kdc[17838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:51:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:51:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:51:13.031581 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:34585
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:38115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:34585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:13.365450 17851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:13.365764 17851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:13.365854 17851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:13.373215 17851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:13.373339 17851 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 15:51:13.373379 17851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:13.373450 17851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:13.373499 17851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:13.373555 17851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:51:13.383872 17851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:38115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:34585
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:34585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:13.385180 17851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:13.386469 17851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:13.398257 17856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:13.398252 17859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:13.398284 17857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:13.399061 17851 server_base.cc:1034] running on GCE node
I20260929 15:51:13.399933 17851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:13.401413 17851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:13.402732 17851 hybrid_clock.cc:648] HybridClock initialized: now 1790697073402655 us; error 44 us; skew 500 ppm
I20260929 15:51:13.403092 17851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:13 dist-test-slave-c51v krb5kdc[17838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697073, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:13.406464 17851 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:51:13.408051 17851 webserver.cc:460] Webserver started at http://127.15.196.254:42779/ using document root <none> and password file <none>
I20260929 15:51:13.408826 17851 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:13.408957 17851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:13.409370 17851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:13.412508 17851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "a5a0767498364440b596e38e61fc7ecc"
format_stamp: "Formatted at 2026-09-29 15:51:13 on dist-test-slave-c51v"
server_key: "b17ea40a58655a5626ed39096ddf454a"
server_key_iv: "405f2422d962dac5ecd813fd8d27132d"
server_key_version: "encryptionkey@0"
I20260929 15:51:13.413297 17851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "a5a0767498364440b596e38e61fc7ecc"
format_stamp: "Formatted at 2026-09-29 15:51:13 on dist-test-slave-c51v"
server_key: "b17ea40a58655a5626ed39096ddf454a"
server_key_iv: "405f2422d962dac5ecd813fd8d27132d"
server_key_version: "encryptionkey@0"
I20260929 15:51:13.418660 17851 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 15:51:13.422299 17866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:13.423748 17851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:13.423961 17851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "a5a0767498364440b596e38e61fc7ecc"
format_stamp: "Formatted at 2026-09-29 15:51:13 on dist-test-slave-c51v"
server_key: "b17ea40a58655a5626ed39096ddf454a"
server_key_iv: "405f2422d962dac5ecd813fd8d27132d"
server_key_version: "encryptionkey@0"
I20260929 15:51:13.424192 17851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:13.471971 17851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:13.476583 17851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:13.476863 17851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:13.497972 17851 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:34585
I20260929 15:51:13.497990 17927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:34585 every 8 connection(s)
I20260929 15:51:13.499380 17851 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:51:13.500985 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 17851
I20260929 15:51:13.501183 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:51:13.501832 16147 external_mini_cluster.cc:1428] Setting key 9b548e20724f707c0cc7132347f56f60
I20260929 15:51:13.503813 17928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 15:51:13 dist-test-slave-c51v krb5kdc[17838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:13.517030 17928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc: Bootstrap starting.
I20260929 15:51:13.521271 17928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:13.522526 17928 log.cc:824] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:13.525882 17928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc: No bootstrap required, opened a new log
I20260929 15:51:13.526150 17931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:13.503870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49968 (local address 127.15.196.254:34585)
0929 15:51:13.504664 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:13.504680 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:13.504729 (+ 49us) server_negotiation.cc:407] Connection header received
0929 15:51:13.506854 (+ 2125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:13.506881 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:13.507452 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:13.507906 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:13.509172 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:13.511266 (+ 2094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:13.512545 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:13.513619 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:13.513816 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:13.517724 (+ 3908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:13.517733 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:13.517753 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:13.517802 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:13.521785 (+ 3983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:13.522475 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:13.522485 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:13.522492 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:13.522695 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:13.523081 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:13.523090 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:13.523095 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:13.523749 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:13.524048 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:13.524429 (+ 381us) server_negotiation.cc:299] Negotiation successful
0929 15:51:13.524849 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":321,"threads_started":1}
I20260929 15:51:13.534417 17928 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5a0767498364440b596e38e61fc7ecc" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34585 } }
I20260929 15:51:13.534801 17928 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:13.534881 17928 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5a0767498364440b596e38e61fc7ecc, State: Initialized, Role: FOLLOWER
I20260929 15:51:13.535406 17928 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5a0767498364440b596e38e61fc7ecc" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34585 } }
I20260929 15:51:13.535606 17928 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:51:13.535701 17928 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:51:13.535884 17928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:13.538264 17928 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5a0767498364440b596e38e61fc7ecc" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34585 } }
I20260929 15:51:13.538705 17928 leader_election.cc:302] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5a0767498364440b596e38e61fc7ecc; no voters:
I20260929 15:51:13.539199 17928 leader_election.cc:288] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:51:13.539354 17933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:13.540063 17933 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [term 1 LEADER]: Becoming Leader. State: Replica: a5a0767498364440b596e38e61fc7ecc, State: Running, Role: LEADER
I20260929 15:51:13.540658 17933 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5a0767498364440b596e38e61fc7ecc" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34585 } }
I20260929 15:51:13.541353 17928 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:13.543166 17935 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5a0767498364440b596e38e61fc7ecc. Latest consensus state: current_term: 1 leader_uuid: "a5a0767498364440b596e38e61fc7ecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5a0767498364440b596e38e61fc7ecc" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34585 } } }
I20260929 15:51:13.543571 17935 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:13.543608 17934 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5a0767498364440b596e38e61fc7ecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5a0767498364440b596e38e61fc7ecc" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34585 } } }
I20260929 15:51:13.543996 17934 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:13.545593 17938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:13.551327 17938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:13.560027 17938 catalog_manager.cc:1347] Generated new cluster ID: c7ed361e4bb94ec885da51d3b1e2a55e
I20260929 15:51:13.560160 17938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:13.567780 17938 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:51:13.569036 17938 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:13.575438 17938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc: Generated new TSK 0
I20260929 15:51:13.576203 17938 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:51:13.624997 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34585
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:38115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:13.961035 17954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:13.961309 17954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:13.961421 17954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:13.968550 17954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:13.968662 17954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:13.968746 17954 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 15:51:13.968799 17954 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 15:51:13.968859 17954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:51:13.980149 17954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:38115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.196.254:34585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:13.981649 17954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:13.983026 17954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:13.997720 17962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:13.997879 17959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:13.997838 17960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:13.999120 17954 server_base.cc:1034] running on GCE node
I20260929 15:51:13.999828 17954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:14.000757 17954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:14.001987 17954 hybrid_clock.cc:648] HybridClock initialized: now 1790697074001941 us; error 34 us; skew 500 ppm
I20260929 15:51:14.002318 17954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:14 dist-test-slave-c51v krb5kdc[17838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697074, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:14.005640 17954 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:51:14.007162 17954 webserver.cc:460] Webserver started at http://127.15.196.193:46165/ using document root <none> and password file <none>
I20260929 15:51:14.007905 17954 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:14.008008 17954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:14.008283 17954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:14.011277 17954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "60d9ce4e736c4bddb9e837ca6428a0f8"
format_stamp: "Formatted at 2026-09-29 15:51:14 on dist-test-slave-c51v"
server_key: "2a3acf81eb3dc94def58a6319593b19b"
server_key_iv: "b3fe64c0560c08aad0817fe1a3b0accb"
server_key_version: "encryptionkey@0"
I20260929 15:51:14.012108 17954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "60d9ce4e736c4bddb9e837ca6428a0f8"
format_stamp: "Formatted at 2026-09-29 15:51:14 on dist-test-slave-c51v"
server_key: "2a3acf81eb3dc94def58a6319593b19b"
server_key_iv: "b3fe64c0560c08aad0817fe1a3b0accb"
server_key_version: "encryptionkey@0"
I20260929 15:51:14.017460 17954 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:51:14.020991 17969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:14.022400 17954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 15:51:14.022581 17954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "60d9ce4e736c4bddb9e837ca6428a0f8"
format_stamp: "Formatted at 2026-09-29 15:51:14 on dist-test-slave-c51v"
server_key: "2a3acf81eb3dc94def58a6319593b19b"
server_key_iv: "b3fe64c0560c08aad0817fe1a3b0accb"
server_key_version: "encryptionkey@0"
I20260929 15:51:14.022790 17954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:14.054677 17954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:14.059274 17954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:14.059645 17954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:14.062378 17954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:14.062520 17954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:14.062657 17954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:14.062753 17954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 15:51:14 dist-test-slave-c51v krb5kdc[17838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697074, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:14.087551 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:14.063082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:34379 (local address 127.15.196.254:34585)
0929 15:51:14.063813 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:14.063820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:14.064955 (+ 1135us) server_negotiation.cc:407] Connection header received
0929 15:51:14.067019 (+ 2064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:14.067025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:14.067114 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:14.067262 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:14.069563 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:14.070876 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:14.074269 (+ 3393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:14.075121 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:14.075323 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:14.079925 (+ 4602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:14.079930 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:14.079934 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:14.079967 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:14.083253 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:14.084314 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:14.084322 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:14.084330 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:14.084451 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:14.085003 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:14.085012 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:14.085017 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:14.085502 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:14.085740 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:14.086537 (+ 797us) server_negotiation.cc:299] Negotiation successful
0929 15:51:14.086926 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":492,"threads_started":1}
I20260929 15:51:14.088177 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:14.063892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:34585 (local address 127.15.196.193:34379)
0929 15:51:14.064766 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:14.064822 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:14.066738 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:14.067533 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:14.067548 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:14.068400 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:14.069304 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:14.069340 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:14.071114 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:14.071122 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:14.074032 (+ 2910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:14.074056 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:14.075417 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:14.075423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:14.075620 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:14.076414 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:14.076436 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:14.079611 (+ 3175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:14.083461 (+ 3850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:14.083475 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:14.083490 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:14.084171 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:14.084569 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:14.084574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:14.084578 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:14.084830 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:14.085686 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:14.085697 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:14.086087 (+ 390us) client_negotiation.cc:769] Sending connection context
0929 15:51:14.086343 (+ 256us) client_negotiation.cc:240] Negotiation successful
0929 15:51:14.086761 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":392,"threads_started":1}
I20260929 15:51:14.109853 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:14.094053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:47005 (local address 127.15.196.254:34585)
0929 15:51:14.094295 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:14.094304 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:14.095160 (+ 856us) server_negotiation.cc:407] Connection header received
0929 15:51:14.095544 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:14.095550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:14.095623 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:14.095705 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:14.097293 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:14.098006 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:14.101023 (+ 3017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:14.101631 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:14.101791 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:14.104465 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:14.104471 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:14.104475 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:14.104516 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:14.107187 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:14.107897 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:14.107903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:14.107907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:14.108039 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:14.108402 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:14.108407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:14.108412 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:14.108660 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:14.108833 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:14.109225 (+ 392us) server_negotiation.cc:299] Negotiation successful
0929 15:51:14.109460 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 15:51:14.109920 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:14.094090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:34585 (local address 127.15.196.193:47005)
0929 15:51:14.095023 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:14.095062 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:14.095373 (+ 311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:14.095786 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:14.095795 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:14.096609 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:14.097129 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:14.097146 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:14.098157 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:14.098164 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:14.100816 (+ 2652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:14.100850 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:14.101869 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:14.101877 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:14.102156 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:14.103009 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:14.103030 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:14.104272 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:14.107352 (+ 3080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:14.107361 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:14.107365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:14.107764 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:14.108152 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:14.108157 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:14.108161 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:14.108288 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:14.108779 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:14.108784 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:14.108975 (+ 191us) client_negotiation.cc:769] Sending connection context
0929 15:51:14.109108 (+ 133us) client_negotiation.cc:240] Negotiation successful
0929 15:51:14.109450 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":502,"threads_started":1}
I20260929 15:51:14.122666 17954 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:34689
I20260929 15:51:14.122684 18089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:34689 every 8 connection(s)
I20260929 15:51:14.124107 17954 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:51:14.126343 18090 heartbeater.cc:347] Connected to a master server at 127.15.196.254:34585
I20260929 15:51:14.126639 18090 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:14.126762 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 17954
I20260929 15:51:14.126981 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:51:14.127589 18090 heartbeater.cc:511] Master 127.15.196.254:34585 requested a full tablet report, sending...
I20260929 15:51:14.127869 16147 external_mini_cluster.cc:1428] Setting key 0010e5abc117e367c5728c1bbfb99bb1
I20260929 15:51:14.130131 17882 ts_manager.cc:194] Registered new tserver with Master: 60d9ce4e736c4bddb9e837ca6428a0f8 (127.15.196.193:34689)
I20260929 15:51:14.131716 17882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:34379
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:51:14.170768 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34585
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:38115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:14.500258 18094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:14.500536 18094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:14.500707 18094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:14.508139 18094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:14.508245 18094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:14.508419 18094 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 15:51:14.508482 18094 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 15:51:14.508620 18094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:51:14.518497 18094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:38115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.196.254:34585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:14.519867 18094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:14.521215 18094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:14.534720 18099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:14.534735 18102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:14.534987 18100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:14.535597 18094 server_base.cc:1034] running on GCE node
I20260929 15:51:14.536217 18094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:14.537161 18094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:14.538391 18094 hybrid_clock.cc:648] HybridClock initialized: now 1790697074538340 us; error 72 us; skew 500 ppm
I20260929 15:51:14.538707 18094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:14 dist-test-slave-c51v krb5kdc[17838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697074, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:14.541967 18094 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:51:14.543467 18094 webserver.cc:460] Webserver started at http://127.15.196.194:44461/ using document root <none> and password file <none>
I20260929 15:51:14.544108 18094 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:14.544207 18094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:14.544478 18094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:14.547570 18094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "769667826c2f47e1b3d103f0a2dd0a06"
format_stamp: "Formatted at 2026-09-29 15:51:14 on dist-test-slave-c51v"
server_key: "dbb4d43b16ee05a9f4cfc7beb189191b"
server_key_iv: "dea5cd1858ce43b0893b6fc24f3ea2da"
server_key_version: "encryptionkey@0"
I20260929 15:51:14.548329 18094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "769667826c2f47e1b3d103f0a2dd0a06"
format_stamp: "Formatted at 2026-09-29 15:51:14 on dist-test-slave-c51v"
server_key: "dbb4d43b16ee05a9f4cfc7beb189191b"
server_key_iv: "dea5cd1858ce43b0893b6fc24f3ea2da"
server_key_version: "encryptionkey@0"
I20260929 15:51:14.553577 18094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:51:14.556986 18109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:14.558645 18094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 15:51:14.558908 18094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "769667826c2f47e1b3d103f0a2dd0a06"
format_stamp: "Formatted at 2026-09-29 15:51:14 on dist-test-slave-c51v"
server_key: "dbb4d43b16ee05a9f4cfc7beb189191b"
server_key_iv: "dea5cd1858ce43b0893b6fc24f3ea2da"
server_key_version: "encryptionkey@0"
I20260929 15:51:14.559177 18094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:14.595762 18094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:14.600080 18094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:14.600386 18094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:14.602933 18094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:14.603067 18094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:14.603269 18094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:14.603420 18094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 15:51:14 dist-test-slave-c51v krb5kdc[17838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697074, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:14.630412 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:14.603932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:36129 (local address 127.15.196.254:34585)
0929 15:51:14.604127 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:14.604133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:14.605685 (+ 1552us) server_negotiation.cc:407] Connection header received
0929 15:51:14.608921 (+ 3236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:14.608929 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:14.609036 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:14.609156 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:14.611761 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:14.612842 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:14.615491 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:14.616082 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:14.616209 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:14.622170 (+ 5961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:14.622179 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:14.622186 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:14.622231 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:14.626194 (+ 3963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:14.627381 (+ 1187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:14.627386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:14.627391 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:14.627541 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:14.628067 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:14.628076 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:14.628082 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:14.628591 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:14.628813 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:14.629586 (+ 773us) server_negotiation.cc:299] Negotiation successful
0929 15:51:14.629900 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 15:51:14.631029 18118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:14.604587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:34585 (local address 127.15.196.194:36129)
0929 15:51:14.605482 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:14.605537 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:14.608538 (+ 3001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:14.609538 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:14.609559 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:14.610624 (+ 1065us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:14.611569 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:14.611601 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:14.613076 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:14.613089 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:14.615346 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:14.615364 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:14.616608 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:14.616621 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:14.616906 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:14.617837 (+ 931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:14.617866 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:14.621846 (+ 3980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:14.626378 (+ 4532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:14.626390 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:14.626413 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:14.627170 (+ 757us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:14.627681 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:14.627685 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:14.627689 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:14.627904 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:14.628760 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:14.628768 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:14.629165 (+ 397us) client_negotiation.cc:769] Sending connection context
0929 15:51:14.629433 (+ 268us) client_negotiation.cc:240] Negotiation successful
0929 15:51:14.629817 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":335,"threads_started":1}
I20260929 15:51:14.651451 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:14.637202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:46667 (local address 127.15.196.254:34585)
0929 15:51:14.637452 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:14.637461 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:14.638018 (+ 557us) server_negotiation.cc:407] Connection header received
0929 15:51:14.638161 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:14.638169 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:14.638269 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:14.638394 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:14.639271 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:14.640541 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:14.643024 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:14.643485 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:14.643629 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:14.645349 (+ 1720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:14.645360 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:14.645363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:14.645391 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:14.648577 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:14.649508 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:14.649515 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:14.649519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:14.649654 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:14.650010 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:14.650016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:14.650019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:14.650340 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:14.650498 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:14.650881 (+ 383us) server_negotiation.cc:299] Negotiation successful
0929 15:51:14.651084 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 15:51:14.651762 18195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:14.637290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:34585 (local address 127.15.196.194:46667)
0929 15:51:14.637927 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:14.637942 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:14.638047 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:14.638405 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:14.638408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:14.638797 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:14.639127 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:14.639135 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:14.640739 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:14.640750 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:14.642883 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:14.642902 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:14.643675 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:14.643681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:14.643852 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:14.644450 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:14.644467 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:14.645167 (+ 700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:14.648719 (+ 3552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:14.648724 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:14.648728 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:14.649369 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:14.649770 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:14.649775 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:14.649778 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:14.649902 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:14.650463 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:14.650468 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:14.650644 (+ 176us) client_negotiation.cc:769] Sending connection context
0929 15:51:14.650784 (+ 140us) client_negotiation.cc:240] Negotiation successful
0929 15:51:14.651239 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":335,"threads_started":1}
I20260929 15:51:14.657301 18094 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:44161
I20260929 15:51:14.657349 18228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:44161 every 8 connection(s)
I20260929 15:51:14.658571 18094 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:51:14.659746 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 18094
I20260929 15:51:14.659919 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:51:14.660446 16147 external_mini_cluster.cc:1428] Setting key f19efe113cc42f83dee5ed949ba33331
I20260929 15:51:14.660611 18229 heartbeater.cc:347] Connected to a master server at 127.15.196.254:34585
I20260929 15:51:14.660790 18229 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:14.661464 18229 heartbeater.cc:511] Master 127.15.196.254:34585 requested a full tablet report, sending...
I20260929 15:51:14.662819 17882 ts_manager.cc:194] Registered new tserver with Master: 769667826c2f47e1b3d103f0a2dd0a06 (127.15.196.194:44161)
I20260929 15:51:14.663808 17882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:36129
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:51:14.702831 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34585
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:38115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:15.027942 18233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:15.028240 18233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:15.028345 18233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:15.036008 18233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:15.036130 18233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:15.036222 18233 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 15:51:15.036276 18233 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 15:51:15.036338 18233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:51:15.046487 18233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:38115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.196.254:34585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:15.048334 18233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:15.049758 18233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:15.064373 18239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:15.064558 18238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:15.065140 18233 server_base.cc:1034] running on GCE node
W20260929 15:51:15.065225 18241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:15.065943 18233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:15.066910 18233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:15.068135 18233 hybrid_clock.cc:648] HybridClock initialized: now 1790697075068095 us; error 50 us; skew 500 ppm
I20260929 15:51:15.068543 18233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:15 dist-test-slave-c51v krb5kdc[17838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697075, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:15.075598 18233 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:51:15.077132 18233 webserver.cc:460] Webserver started at http://127.15.196.195:43379/ using document root <none> and password file <none>
I20260929 15:51:15.077873 18233 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:15.077992 18233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:15.078286 18233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:15.081664 18233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "8d036942aa3349c88175e94143685d9a"
format_stamp: "Formatted at 2026-09-29 15:51:15 on dist-test-slave-c51v"
server_key: "16dd3b589b62d0ec1a9b3e10aa9da0c4"
server_key_iv: "6c5b6dc292b1f876dd5792f29bad9d43"
server_key_version: "encryptionkey@0"
I20260929 15:51:15.082468 18233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "8d036942aa3349c88175e94143685d9a"
format_stamp: "Formatted at 2026-09-29 15:51:15 on dist-test-slave-c51v"
server_key: "16dd3b589b62d0ec1a9b3e10aa9da0c4"
server_key_iv: "6c5b6dc292b1f876dd5792f29bad9d43"
server_key_version: "encryptionkey@0"
I20260929 15:51:15.088218 18233 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 15:51:15.092344 18248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:15.093752 18233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:51:15.093909 18233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "8d036942aa3349c88175e94143685d9a"
format_stamp: "Formatted at 2026-09-29 15:51:15 on dist-test-slave-c51v"
server_key: "16dd3b589b62d0ec1a9b3e10aa9da0c4"
server_key_iv: "6c5b6dc292b1f876dd5792f29bad9d43"
server_key_version: "encryptionkey@0"
I20260929 15:51:15.094094 18233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:15.117885 18233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:15.121966 18233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:15.122301 18233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:15.125204 18233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:15.125388 18233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260929 15:51:15.125519 18233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:15.125605 18233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:15.135365 18090 heartbeater.cc:503] Master 127.15.196.254:34585 was elected leader, sending a full tablet report...
Sep 29 15:51:15 dist-test-slave-c51v krb5kdc[17838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697075, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:15.149441 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.126022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:57481 (local address 127.15.196.254:34585)
0929 15:51:15.126229 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.126235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:15.128112 (+ 1877us) server_negotiation.cc:407] Connection header received
0929 15:51:15.130446 (+ 2334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:15.130452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:15.130540 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:15.130658 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:15.132564 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.133445 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.136490 (+ 3045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.137129 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.137242 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.142027 (+ 4785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:15.142032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:15.142036 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:15.142066 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:15.145324 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:15.146409 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:15.146417 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:15.146421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:15.146584 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:15.147194 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:15.147201 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:15.147206 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:15.147619 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:15.147789 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:15.148832 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0929 15:51:15.149061 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 15:51:15.150570 18257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.126981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:34585 (local address 127.15.196.195:57481)
0929 15:51:15.127956 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:15.128008 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:15.130153 (+ 2145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:15.130919 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:15.130927 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:15.131537 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:15.132301 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.132340 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.133601 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.133608 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.136291 (+ 2683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.136305 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.137554 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.137559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.137754 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.138644 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:15.138664 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:15.141763 (+ 3099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:15.145476 (+ 3713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:15.145486 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:15.145502 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:15.146210 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:15.146685 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:15.146691 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:15.146694 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:15.147013 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:15.147878 (+ 865us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:15.147892 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:15.148381 (+ 489us) client_negotiation.cc:769] Sending connection context
0929 15:51:15.148724 (+ 343us) client_negotiation.cc:240] Negotiation successful
0929 15:51:15.149303 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":447,"threads_started":1}
I20260929 15:51:15.170674 18309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.156647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:34585 (local address 127.15.196.195:43425)
0929 15:51:15.157317 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:15.157341 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:15.157522 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:15.157874 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:15.157879 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:15.158575 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:15.159082 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.159093 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.160436 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.160441 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.161991 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.162005 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.162766 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.162772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.162985 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.163649 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:15.163665 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:15.164351 (+ 686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:15.167446 (+ 3095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:15.167455 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:15.167459 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:15.168039 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:15.168504 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:15.168511 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:15.168517 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:15.168705 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:15.169378 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:15.169383 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:15.169704 (+ 321us) client_negotiation.cc:769] Sending connection context
0929 15:51:15.169926 (+ 222us) client_negotiation.cc:240] Negotiation successful
0929 15:51:15.170330 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":350,"threads_started":1}
I20260929 15:51:15.171501 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.156611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:43425 (local address 127.15.196.254:34585)
0929 15:51:15.156792 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.156799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:15.157450 (+ 651us) server_negotiation.cc:407] Connection header received
0929 15:51:15.157668 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:15.157673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:15.157742 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:15.157848 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:15.159232 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.160296 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.162154 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.162581 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.162723 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.164545 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:15.164555 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:15.164559 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:15.164588 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:15.167270 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:15.168208 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:15.168212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:15.168214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:15.168333 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:15.168835 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:15.168843 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:15.168849 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:15.169265 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:15.169427 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:15.171022 (+ 1595us) server_negotiation.cc:299] Negotiation successful
0929 15:51:15.171190 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 15:51:15.186779 18233 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:45567
I20260929 15:51:15.186796 18367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:45567 every 8 connection(s)
I20260929 15:51:15.188130 18233 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:51:15.190347 18368 heartbeater.cc:347] Connected to a master server at 127.15.196.254:34585
I20260929 15:51:15.190583 18368 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:15.190883 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 18233
I20260929 15:51:15.191062 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:51:15.191488 18368 heartbeater.cc:511] Master 127.15.196.254:34585 requested a full tablet report, sending...
I20260929 15:51:15.191651 16147 external_mini_cluster.cc:1428] Setting key 3cf71172b148fac630b1143a80b78aee
I20260929 15:51:15.192962 17882 ts_manager.cc:194] Registered new tserver with Master: 8d036942aa3349c88175e94143685d9a (127.15.196.195:45567)
I20260929 15:51:15.194053 17882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:57481
I20260929 15:51:15.195016 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 15:51:15 dist-test-slave-c51v krb5kdc[17838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 15:51:15 dist-test-slave-c51v krb5kdc[17838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:15.241506 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.227350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49978 (local address 127.15.196.254:34585)
0929 15:51:15.227607 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.227616 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:15.227944 (+ 328us) server_negotiation.cc:407] Connection header received
0929 15:51:15.228160 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:15.228165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:15.228237 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:15.228346 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:15.229812 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.230554 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.231888 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.232311 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.232445 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.236342 (+ 3897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:15.236354 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:15.236358 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:15.236393 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:15.238741 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:15.239453 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:15.239458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:15.239462 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:15.239604 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:15.240000 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:15.240006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:15.240010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:15.240281 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:15.240542 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:15.240884 (+ 342us) server_negotiation.cc:299] Negotiation successful
0929 15:51:15.241148 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 15:51:15.245460 17883 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:51:15.252494 17883 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:51:15.288596 18386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.267217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44628 (local address 127.15.196.194:44161)
0929 15:51:15.281017 (+ 13800us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.281027 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:15.281056 (+ 29us) server_negotiation.cc:407] Connection header received
0929 15:51:15.281164 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:15.281176 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:15.281512 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:15.281756 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:15.282579 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.283884 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.285935 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.287388 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.287561 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.287766 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:15.288112 (+ 346us) server_negotiation.cc:299] Negotiation successful
0929 15:51:15.288292 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13632,"thread_start_us":13476,"threads_started":1}
I20260929 15:51:15.288622 18385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.266911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:44161 (local address 127.0.0.1:44628)
0929 15:51:15.280296 (+ 13385us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:15.280327 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:15.280485 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:15.281832 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:15.281845 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:15.281894 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:15.282341 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.282367 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.284023 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.284032 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.285727 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.285744 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.287615 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.287621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.287849 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.287905 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 15:51:15.288054 (+ 149us) client_negotiation.cc:240] Negotiation successful
0929 15:51:15.288244 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13206,"thread_start_us":14073,"threads_started":1}
I20260929 15:51:15.290446 18382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.265427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43918 (local address 127.15.196.193:34689)
0929 15:51:15.282337 (+ 16910us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.282348 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:15.282380 (+ 32us) server_negotiation.cc:407] Connection header received
0929 15:51:15.282479 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:15.282490 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:15.282713 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:15.283626 (+ 913us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:15.283820 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.285261 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.287851 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.289309 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.289515 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.289685 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:15.289926 (+ 241us) server_negotiation.cc:299] Negotiation successful
0929 15:51:15.290100 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16754,"thread_start_us":16626,"threads_started":1}
I20260929 15:51:15.291062 18383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.265879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:45567 (local address 127.0.0.1:53480)
0929 15:51:15.276090 (+ 10211us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:15.276125 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:15.276286 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:15.282622 (+ 6336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:15.282627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:15.282650 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:15.283119 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.283136 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.285315 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.285320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.286942 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.286953 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.290267 (+ 3314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.290273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.290450 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.290492 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 15:51:15.290591 (+ 99us) client_negotiation.cc:240] Negotiation successful
0929 15:51:15.290728 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10012,"thread_start_us":9842,"threads_started":1}
I20260929 15:51:15.291465 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.266198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53480 (local address 127.15.196.195:45567)
0929 15:51:15.281664 (+ 15466us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.281674 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:15.281708 (+ 34us) server_negotiation.cc:407] Connection header received
0929 15:51:15.281801 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:15.281811 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:15.282089 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:15.282295 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:15.283417 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.285153 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.287106 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.289738 (+ 2632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.289974 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.290132 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:15.290879 (+ 747us) server_negotiation.cc:299] Negotiation successful
0929 15:51:15.291089 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15272,"thread_start_us":15107,"threads_started":1}
I20260929 15:51:15.292708 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.265276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34689 (local address 127.0.0.1:43918)
0929 15:51:15.266516 (+ 1240us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:15.266564 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:15.266719 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:15.282959 (+ 16240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:15.282966 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:15.282997 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:15.283415 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.283427 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.285423 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.285427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.287661 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.287674 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.289539 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.289545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.289704 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.289735 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 15:51:15.289828 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 15:51:15.290009 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":990,"mutex_wait_us":79,"thread_start_us":859,"threads_started":1}
I20260929 15:51:15.293174 18159 tablet_service.cc:1467] Processing CreateTablet for tablet 46e34af9345e45ef91ddec8ba9924ca5 (DEFAULT_TABLE table=test-table [id=7577ae33aec44b269737633c25724aed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:15.294682 18159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46e34af9345e45ef91ddec8ba9924ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:15.295188 18298 tablet_service.cc:1467] Processing CreateTablet for tablet 46e34af9345e45ef91ddec8ba9924ca5 (DEFAULT_TABLE table=test-table [id=7577ae33aec44b269737633c25724aed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:15.296662 18298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46e34af9345e45ef91ddec8ba9924ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:15.296380 18020 tablet_service.cc:1467] Processing CreateTablet for tablet 46e34af9345e45ef91ddec8ba9924ca5 (DEFAULT_TABLE table=test-table [id=7577ae33aec44b269737633c25724aed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:15.297680 18020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46e34af9345e45ef91ddec8ba9924ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:15.307891 18387 tablet_bootstrap.cc:492] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a: Bootstrap starting.
I20260929 15:51:15.311046 18387 tablet_bootstrap.cc:654] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:15.311930 18388 tablet_bootstrap.cc:492] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06: Bootstrap starting.
I20260929 15:51:15.312413 18387 log.cc:824] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:15.313709 18389 tablet_bootstrap.cc:492] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8: Bootstrap starting.
I20260929 15:51:15.316375 18388 tablet_bootstrap.cc:654] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:15.316767 18389 tablet_bootstrap.cc:654] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:15.317802 18389 log.cc:824] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:15.318046 18388 log.cc:824] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:15.321172 18389 tablet_bootstrap.cc:492] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8: No bootstrap required, opened a new log
I20260929 15:51:15.321343 18387 tablet_bootstrap.cc:492] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a: No bootstrap required, opened a new log
I20260929 15:51:15.321344 18388 tablet_bootstrap.cc:492] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06: No bootstrap required, opened a new log
I20260929 15:51:15.321489 18389 ts_tablet_manager.cc:1397] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 15:51:15.321609 18387 ts_tablet_manager.cc:1397] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260929 15:51:15.321609 18388 ts_tablet_manager.cc:1397] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260929 15:51:15.329602 18389 raft_consensus.cc:348] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.330368 18389 raft_consensus.cc:374] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:15.330485 18389 raft_consensus.cc:729] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60d9ce4e736c4bddb9e837ca6428a0f8, State: Initialized, Role: FOLLOWER
I20260929 15:51:15.331094 18389 consensus_queue.cc:260] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.332252 18387 raft_consensus.cc:348] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.332209 18388 raft_consensus.cc:348] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.332674 18388 raft_consensus.cc:374] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:15.332710 18387 raft_consensus.cc:374] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:15.332798 18387 raft_consensus.cc:729] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d036942aa3349c88175e94143685d9a, State: Initialized, Role: FOLLOWER
I20260929 15:51:15.332849 18389 ts_tablet_manager.cc:1428] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260929 15:51:15.332857 18388 raft_consensus.cc:729] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 769667826c2f47e1b3d103f0a2dd0a06, State: Initialized, Role: FOLLOWER
I20260929 15:51:15.333516 18387 consensus_queue.cc:260] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.333549 18388 consensus_queue.cc:260] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.334627 18368 heartbeater.cc:503] Master 127.15.196.254:34585 was elected leader, sending a full tablet report...
I20260929 15:51:15.335040 18229 heartbeater.cc:503] Master 127.15.196.254:34585 was elected leader, sending a full tablet report...
I20260929 15:51:15.335320 18387 ts_tablet_manager.cc:1428] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260929 15:51:15.336732 18388 ts_tablet_manager.cc:1428] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
W20260929 15:51:15.377187 18091 tablet.cc:2369] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 15:51:15.410570 18230 tablet.cc:2369] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:51:15.426002 18393 raft_consensus.cc:482] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:51:15.426352 18393 raft_consensus.cc:504] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.428354 18393 leader_election.cc:288] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 769667826c2f47e1b3d103f0a2dd0a06 (127.15.196.194:44161), 8d036942aa3349c88175e94143685d9a (127.15.196.195:45567)
I20260929 15:51:15.435271 18386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.428629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:58847 (local address 127.15.196.194:44161)
0929 15:51:15.428852 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.428861 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:15.429524 (+ 663us) server_negotiation.cc:407] Connection header received
0929 15:51:15.429746 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:15.429753 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:15.429832 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:15.429928 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:15.430587 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.431399 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.433426 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.434285 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.434463 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.434556 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:15.434873 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 15:51:15.435003 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 15:51:15.435278 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.429157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:41283 (local address 127.15.196.195:45567)
0929 15:51:15.429388 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.429395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:15.429765 (+ 370us) server_negotiation.cc:407] Connection header received
0929 15:51:15.429966 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:15.429971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:15.430048 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:15.430143 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:15.430740 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.431509 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.433381 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.434299 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.434494 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.434566 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:15.434871 (+ 305us) server_negotiation.cc:299] Negotiation successful
0929 15:51:15.434998 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 15:51:15.435463 18397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.428798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:45567 (local address 127.15.196.193:41283)
0929 15:51:15.429639 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:15.429673 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:15.429799 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:15.430172 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:15.430175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:15.430194 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:15.430585 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.430595 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.431644 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.431648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.433240 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.433251 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.434553 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.434559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.434705 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.434731 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 15:51:15.434843 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 15:51:15.434984 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":42,"thread_start_us":490,"threads_started":1}
I20260929 15:51:15.435525 18396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.428580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:44161 (local address 127.15.196.193:58847)
0929 15:51:15.429382 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:15.429408 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:15.429582 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:15.429959 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:15.429963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:15.429989 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:15.430438 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.430449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.431525 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.431530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.433231 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.433248 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.434511 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.434518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.434691 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.434724 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 15:51:15.434836 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 15:51:15.434997 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":59,"thread_start_us":373,"threads_started":1}
I20260929 15:51:15.436559 18179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46e34af9345e45ef91ddec8ba9924ca5" candidate_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "769667826c2f47e1b3d103f0a2dd0a06" is_pre_election: true
I20260929 15:51:15.436582 18323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46e34af9345e45ef91ddec8ba9924ca5" candidate_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d036942aa3349c88175e94143685d9a" is_pre_election: true
I20260929 15:51:15.437052 18179 raft_consensus.cc:2454] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60d9ce4e736c4bddb9e837ca6428a0f8 in term 0.
I20260929 15:51:15.437068 18323 raft_consensus.cc:2454] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60d9ce4e736c4bddb9e837ca6428a0f8 in term 0.
I20260929 15:51:15.437733 17973 leader_election.cc:302] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 60d9ce4e736c4bddb9e837ca6428a0f8, 769667826c2f47e1b3d103f0a2dd0a06, 8d036942aa3349c88175e94143685d9a; no voters:
I20260929 15:51:15.438127 18393 raft_consensus.cc:2781] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:51:15.438267 18393 raft_consensus.cc:482] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:51:15.438400 18393 raft_consensus.cc:3037] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Advancing to term 1
W20260929 15:51:15.439625 18369 tablet.cc:2369] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:51:15.440744 18393 raft_consensus.cc:504] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.441363 18393 leader_election.cc:288] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [CANDIDATE]: Term 1 election: Requested vote from peers 769667826c2f47e1b3d103f0a2dd0a06 (127.15.196.194:44161), 8d036942aa3349c88175e94143685d9a (127.15.196.195:45567)
I20260929 15:51:15.441906 18179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46e34af9345e45ef91ddec8ba9924ca5" candidate_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "769667826c2f47e1b3d103f0a2dd0a06"
I20260929 15:51:15.441915 18323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46e34af9345e45ef91ddec8ba9924ca5" candidate_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d036942aa3349c88175e94143685d9a"
I20260929 15:51:15.442121 18323 raft_consensus.cc:3037] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:15.442107 18179 raft_consensus.cc:3037] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:15.444391 18323 raft_consensus.cc:2454] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60d9ce4e736c4bddb9e837ca6428a0f8 in term 1.
I20260929 15:51:15.444406 18179 raft_consensus.cc:2454] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60d9ce4e736c4bddb9e837ca6428a0f8 in term 1.
I20260929 15:51:15.444861 17972 leader_election.cc:302] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60d9ce4e736c4bddb9e837ca6428a0f8, 8d036942aa3349c88175e94143685d9a; no voters:
I20260929 15:51:15.445218 18393 raft_consensus.cc:2781] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:15.445698 18393 raft_consensus.cc:686] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 LEADER]: Becoming Leader. State: Replica: 60d9ce4e736c4bddb9e837ca6428a0f8, State: Running, Role: LEADER
I20260929 15:51:15.446231 18393 consensus_queue.cc:237] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.450753 17883 catalog_manager.cc:5495] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60d9ce4e736c4bddb9e837ca6428a0f8 (127.15.196.193). New cstate: current_term: 1 leader_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } health_report { overall_health: UNKNOWN } } }
Sep 29 15:51:15 dist-test-slave-c51v krb5kdc[17838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697073, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:15.495378 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.481664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49994 (local address 127.15.196.254:34585)
0929 15:51:15.481876 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.481887 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:15.482583 (+ 696us) server_negotiation.cc:407] Connection header received
0929 15:51:15.482720 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:15.482724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:15.482796 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:15.482895 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:15.484117 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.484859 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.486038 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.486453 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.486543 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.489839 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:15.489844 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:15.489847 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:15.489919 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:15.492560 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:15.493272 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:15.493277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:15.493285 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:15.493404 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:15.493770 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:15.493777 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:15.493780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:15.494045 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:15.494193 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:15.494618 (+ 425us) server_negotiation.cc:299] Negotiation successful
0929 15:51:15.494949 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":137,"server-negotiator.queue_time_us":68}
I20260929 15:51:15.495399 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.481740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:34585 (local address 127.0.0.1:49994)
0929 15:51:15.482466 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:15.482491 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:15.482618 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:15.482937 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:15.482940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:15.483708 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:15.483990 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.483999 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.484969 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.484972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.485908 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.485918 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.486607 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.486612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.486731 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.487457 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:15.487475 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:15.489646 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:15.492695 (+ 3049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:15.492712 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:15.492721 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:15.493142 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:15.493510 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:15.493517 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:15.493520 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:15.493667 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:15.494174 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:15.494182 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:15.494349 (+ 167us) client_negotiation.cc:769] Sending connection context
0929 15:51:15.494514 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 15:51:15.494815 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":326,"threads_started":1}
I20260929 15:51:15.500921 17883 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:49994:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 15:51:15.502039 17883 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:51:15.509128 18159 tablet_service.cc:1467] Processing CreateTablet for tablet a0c5817c7fd4499d9169cda33b0943f5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9084baff1f284b219ebb88d6fb089152]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 15:51:15.509109 18020 tablet_service.cc:1467] Processing CreateTablet for tablet a0c5817c7fd4499d9169cda33b0943f5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9084baff1f284b219ebb88d6fb089152]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 15:51:15.509265 18298 tablet_service.cc:1467] Processing CreateTablet for tablet a0c5817c7fd4499d9169cda33b0943f5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9084baff1f284b219ebb88d6fb089152]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 15:51:15.510123 18020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0c5817c7fd4499d9169cda33b0943f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:15.510123 18298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0c5817c7fd4499d9169cda33b0943f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:15.510123 18159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0c5817c7fd4499d9169cda33b0943f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:15.517206 18389 tablet_bootstrap.cc:492] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8: Bootstrap starting.
I20260929 15:51:15.517282 18388 tablet_bootstrap.cc:492] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06: Bootstrap starting.
I20260929 15:51:15.517304 18387 tablet_bootstrap.cc:492] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a: Bootstrap starting.
I20260929 15:51:15.518604 18388 tablet_bootstrap.cc:654] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:15.518874 18387 tablet_bootstrap.cc:654] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:15.518868 18389 tablet_bootstrap.cc:654] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:15.520715 18388 tablet_bootstrap.cc:492] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06: No bootstrap required, opened a new log
I20260929 15:51:15.520887 18388 ts_tablet_manager.cc:1397] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260929 15:51:15.521499 18388 raft_consensus.cc:348] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.521742 18388 raft_consensus.cc:374] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:15.521831 18388 raft_consensus.cc:729] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 769667826c2f47e1b3d103f0a2dd0a06, State: Initialized, Role: FOLLOWER
I20260929 15:51:15.522094 18388 consensus_queue.cc:260] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.522567 18395 tablet_replica.cc:439] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } } }
I20260929 15:51:15.522928 18395 tablet_replica.cc:442] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:51:15.523934 18388 ts_tablet_manager.cc:1428] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 15:51:15.529233 18387 tablet_bootstrap.cc:492] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a: No bootstrap required, opened a new log
I20260929 15:51:15.529402 18387 ts_tablet_manager.cc:1397] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.009s
I20260929 15:51:15.530046 18387 raft_consensus.cc:348] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.530309 18387 raft_consensus.cc:374] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:15.530417 18387 raft_consensus.cc:729] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d036942aa3349c88175e94143685d9a, State: Initialized, Role: FOLLOWER
I20260929 15:51:15.531025 18387 consensus_queue.cc:260] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.531690 18387 ts_tablet_manager.cc:1428] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 15:51:15.531560 18394 tablet_replica.cc:439] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } } }
I20260929 15:51:15.531937 18394 tablet_replica.cc:442] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:51:15.533730 18389 tablet_bootstrap.cc:492] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8: No bootstrap required, opened a new log
I20260929 15:51:15.533977 18389 ts_tablet_manager.cc:1397] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.004s
I20260929 15:51:15.534847 18389 raft_consensus.cc:348] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.535051 18389 raft_consensus.cc:374] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:15.535141 18389 raft_consensus.cc:729] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60d9ce4e736c4bddb9e837ca6428a0f8, State: Initialized, Role: FOLLOWER
I20260929 15:51:15.535426 18389 consensus_queue.cc:260] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.535889 18393 tablet_replica.cc:439] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } } }
I20260929 15:51:15.536103 18389 ts_tablet_manager.cc:1428] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 15:51:15.536162 18393 tablet_replica.cc:442] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:51:15.603873 18395 raft_consensus.cc:482] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:51:15.604126 18395 raft_consensus.cc:504] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.605849 18395 leader_election.cc:288] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60d9ce4e736c4bddb9e837ca6428a0f8 (127.15.196.193:34689), 8d036942aa3349c88175e94143685d9a (127.15.196.195:45567)
I20260929 15:51:15.612836 18382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.606135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:50653 (local address 127.15.196.193:34689)
0929 15:51:15.606369 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.606376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:15.607241 (+ 865us) server_negotiation.cc:407] Connection header received
0929 15:51:15.607483 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:15.607489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:15.607580 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:15.607647 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:15.608277 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.609025 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.611025 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.611864 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.612033 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.612127 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:15.612471 (+ 344us) server_negotiation.cc:299] Negotiation successful
0929 15:51:15.612592 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 15:51:15.612885 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.606317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:45547 (local address 127.15.196.195:45567)
0929 15:51:15.606518 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.606526 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:15.607332 (+ 806us) server_negotiation.cc:407] Connection header received
0929 15:51:15.607544 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:15.607548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:15.607604 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:15.607713 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:15.608348 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.609112 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.611076 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.611871 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.612052 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.612144 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:15.612440 (+ 296us) server_negotiation.cc:299] Negotiation successful
0929 15:51:15.612566 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 15:51:15.612991 18410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.606281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:45567 (local address 127.15.196.194:45547)
0929 15:51:15.607174 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:15.607212 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:15.607407 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:15.607748 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:15.607751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:15.607776 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:15.608162 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.608171 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.609274 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.609278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.610901 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.610912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.612132 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.612140 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.612287 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.612311 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 15:51:15.612386 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 15:51:15.612542 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"mutex_wait_us":21,"thread_start_us":613,"threads_started":1}
I20260929 15:51:15.613255 18409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.605999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34689 (local address 127.15.196.194:50653)
0929 15:51:15.607127 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:15.607143 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:15.607328 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:15.607711 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:15.607716 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:15.607739 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:15.608118 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.608131 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.609156 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.609163 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.610877 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.610890 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.612118 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.612124 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.612292 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.612322 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 15:51:15.612426 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 15:51:15.612554 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"mutex_wait_us":102,"thread_start_us":774,"threads_started":1}
I20260929 15:51:15.613747 18323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0c5817c7fd4499d9169cda33b0943f5" candidate_uuid: "769667826c2f47e1b3d103f0a2dd0a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d036942aa3349c88175e94143685d9a" is_pre_election: true
I20260929 15:51:15.614033 18323 raft_consensus.cc:2454] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 769667826c2f47e1b3d103f0a2dd0a06 in term 0.
I20260929 15:51:15.614385 18045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0c5817c7fd4499d9169cda33b0943f5" candidate_uuid: "769667826c2f47e1b3d103f0a2dd0a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" is_pre_election: true
I20260929 15:51:15.614491 18112 leader_election.cc:302] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 769667826c2f47e1b3d103f0a2dd0a06, 8d036942aa3349c88175e94143685d9a; no voters:
I20260929 15:51:15.614810 18045 raft_consensus.cc:2454] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 769667826c2f47e1b3d103f0a2dd0a06 in term 0.
I20260929 15:51:15.614912 18395 raft_consensus.cc:2781] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:51:15.615021 18395 raft_consensus.cc:482] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:51:15.615130 18395 raft_consensus.cc:3037] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:15.617304 18395 raft_consensus.cc:504] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.617905 18395 leader_election.cc:288] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [CANDIDATE]: Term 1 election: Requested vote from peers 60d9ce4e736c4bddb9e837ca6428a0f8 (127.15.196.193:34689), 8d036942aa3349c88175e94143685d9a (127.15.196.195:45567)
I20260929 15:51:15.618422 18323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0c5817c7fd4499d9169cda33b0943f5" candidate_uuid: "769667826c2f47e1b3d103f0a2dd0a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d036942aa3349c88175e94143685d9a"
I20260929 15:51:15.618431 18045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0c5817c7fd4499d9169cda33b0943f5" candidate_uuid: "769667826c2f47e1b3d103f0a2dd0a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8"
I20260929 15:51:15.618656 18323 raft_consensus.cc:3037] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:15.618646 18045 raft_consensus.cc:3037] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:15.620729 18323 raft_consensus.cc:2454] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 769667826c2f47e1b3d103f0a2dd0a06 in term 1.
I20260929 15:51:15.620838 18045 raft_consensus.cc:2454] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 769667826c2f47e1b3d103f0a2dd0a06 in term 1.
I20260929 15:51:15.621232 18112 leader_election.cc:302] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 769667826c2f47e1b3d103f0a2dd0a06, 8d036942aa3349c88175e94143685d9a; no voters:
I20260929 15:51:15.621569 18395 raft_consensus.cc:2781] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:15.622017 18395 raft_consensus.cc:686] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 LEADER]: Becoming Leader. State: Replica: 769667826c2f47e1b3d103f0a2dd0a06, State: Running, Role: LEADER
I20260929 15:51:15.622552 18395 consensus_queue.cc:237] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.626513 18411 tablet_replica.cc:439] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06: TxnStatusTablet state changed. Reason: New leader 769667826c2f47e1b3d103f0a2dd0a06. Latest consensus state: current_term: 1 leader_uuid: "769667826c2f47e1b3d103f0a2dd0a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } } }
I20260929 15:51:15.626897 18411 tablet_replica.cc:442] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06: This TxnStatusTablet replica's current role is: LEADER
I20260929 15:51:15.627701 18413 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 15:51:15.628014 18413 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 15:51:15.628254 17882 catalog_manager.cc:5495] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 769667826c2f47e1b3d103f0a2dd0a06 (127.15.196.194). New cstate: current_term: 1 leader_uuid: "769667826c2f47e1b3d103f0a2dd0a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } health_report { overall_health: UNKNOWN } } }
I20260929 15:51:15.664252 18386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.656491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44632 (local address 127.15.196.194:44161)
0929 15:51:15.656696 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.656702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:15.657149 (+ 447us) server_negotiation.cc:407] Connection header received
0929 15:51:15.657248 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:15.657254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:15.657379 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:15.657545 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:51:15.657979 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.659240 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.661303 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.661945 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.662147 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.662690 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:51:15.663445 (+ 755us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:51:15.663582 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:15.663837 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 15:51:15.663923 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 15:51:15.664314 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.656603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:44161 (local address 127.0.0.1:44632)
0929 15:51:15.656815 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:15.656836 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:15.656978 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:15.657582 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:15.657588 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:15.657607 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 15:51:15.657812 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.657825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.659455 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.659461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.661163 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.661179 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.662128 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.662135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.662366 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.662440 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 15:51:15.663570 (+ 1130us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 15:51:15.663637 (+ 67us) client_negotiation.cc:769] Sending connection context
0929 15:51:15.663792 (+ 155us) client_negotiation.cc:240] Negotiation successful
0929 15:51:15.663950 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 15:51:15.673350 18045 raft_consensus.cc:1261] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Refusing update from remote peer 769667826c2f47e1b3d103f0a2dd0a06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:15.674528 18323 raft_consensus.cc:1261] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a [term 1 FOLLOWER]: Refusing update from remote peer 769667826c2f47e1b3d103f0a2dd0a06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:15.675160 18411 consensus_queue.cc:1035] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:15.675493 18395 consensus_queue.cc:1035] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:15.690779 18393 tablet_replica.cc:439] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8: TxnStatusTablet state changed. Reason: New leader 769667826c2f47e1b3d103f0a2dd0a06. Latest consensus state: current_term: 1 leader_uuid: "769667826c2f47e1b3d103f0a2dd0a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } } }
I20260929 15:51:15.691169 18393 tablet_replica.cc:442] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:51:15.691730 18394 tablet_replica.cc:439] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a: TxnStatusTablet state changed. Reason: New leader 769667826c2f47e1b3d103f0a2dd0a06. Latest consensus state: current_term: 1 leader_uuid: "769667826c2f47e1b3d103f0a2dd0a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } } }
I20260929 15:51:15.692008 18394 tablet_replica.cc:442] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:51:15.694072 18416 mvcc.cc:205] Tried to move back new op lower bound from 7334695221947854848 to 7334695221754597376. Current Snapshot: MvccSnapshot[applied={T|T < 7334695221947854848}]
I20260929 15:51:15.695181 18411 tablet_replica.cc:439] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "769667826c2f47e1b3d103f0a2dd0a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } } }
I20260929 15:51:15.695551 18411 tablet_replica.cc:442] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06: This TxnStatusTablet replica's current role is: LEADER
I20260929 15:51:15.695782 18411 tablet_replica.cc:439] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "769667826c2f47e1b3d103f0a2dd0a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } } }
I20260929 15:51:15.696087 18411 tablet_replica.cc:442] T a0c5817c7fd4499d9169cda33b0943f5 P 769667826c2f47e1b3d103f0a2dd0a06: This TxnStatusTablet replica's current role is: LEADER
I20260929 15:51:15.705431 18394 tablet_replica.cc:439] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "769667826c2f47e1b3d103f0a2dd0a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } } }
I20260929 15:51:15.705714 18394 tablet_replica.cc:442] T a0c5817c7fd4499d9169cda33b0943f5 P 8d036942aa3349c88175e94143685d9a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:51:15.714039 18393 tablet_replica.cc:439] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "769667826c2f47e1b3d103f0a2dd0a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } } }
I20260929 15:51:15.714361 18393 tablet_replica.cc:442] T a0c5817c7fd4499d9169cda33b0943f5 P 60d9ce4e736c4bddb9e837ca6428a0f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:51:15.721514 18382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.712667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43920 (local address 127.15.196.193:34689)
0929 15:51:15.712874 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.712885 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:15.715072 (+ 2187us) server_negotiation.cc:407] Connection header received
0929 15:51:15.715463 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:15.715473 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:15.715611 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:15.715797 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:51:15.716570 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.717837 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.719198 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.719669 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.719827 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.720281 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:51:15.720853 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:51:15.721009 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:15.721117 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 15:51:15.721201 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260929 15:51:15.738927 18386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.733235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:52039 (local address 127.15.196.194:44161)
0929 15:51:15.733475 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.733483 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:15.733888 (+ 405us) server_negotiation.cc:407] Connection header received
0929 15:51:15.734179 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:15.734199 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:15.734276 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:15.734387 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:51:15.734887 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.735742 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.737083 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.737548 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.737714 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.738171 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:51:15.738277 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:51:15.738384 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:15.738534 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 15:51:15.738613 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 15:51:15.739008 18428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.733139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:44161 (local address 127.15.196.193:52039)
0929 15:51:15.733765 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:15.733782 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:15.734002 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:15.734404 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:15.734408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:15.734420 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 15:51:15.734733 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.734747 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.735875 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.735883 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.736938 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.736949 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.737769 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.737777 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.737907 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.737960 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 15:51:15.738381 (+ 421us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 15:51:15.738403 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 15:51:15.738500 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 15:51:15.738633 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":340,"threads_started":1}
I20260929 15:51:15.749293 18323 raft_consensus.cc:1261] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a [term 1 FOLLOWER]: Refusing update from remote peer 60d9ce4e736c4bddb9e837ca6428a0f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:15.749728 18179 raft_consensus.cc:1261] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Refusing update from remote peer 60d9ce4e736c4bddb9e837ca6428a0f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:15.750540 18398 consensus_queue.cc:1035] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:15.750900 18393 consensus_queue.cc:1035] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:15.757042 18418 mvcc.cc:205] Tried to move back new op lower bound from 7334695222263353344 to 7334695221032017920. Current Snapshot: MvccSnapshot[applied={T|T < 7334695222263353344}]
I20260929 15:51:15.791923 18382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.785059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:35713 (local address 127.15.196.193:34689)
0929 15:51:15.785267 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.785275 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:15.785761 (+ 486us) server_negotiation.cc:407] Connection header received
0929 15:51:15.785931 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:15.785936 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:15.786012 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:15.786099 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:51:15.786596 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.787764 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.789689 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.790359 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.790485 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.790999 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:51:15.791154 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:51:15.791264 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:15.791524 (+ 260us) server_negotiation.cc:299] Negotiation successful
0929 15:51:15.791605 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 15:51:15.792061 18432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.784998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34689 (local address 127.15.196.194:35713)
0929 15:51:15.785658 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:15.785679 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:15.785796 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:15.786122 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:15.786125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:15.786133 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 15:51:15.786432 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.786440 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.787894 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.787898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.789552 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.789565 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.790567 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.790574 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.790747 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.790841 (+ 94us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 15:51:15.791367 (+ 526us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 15:51:15.791378 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 15:51:15.791496 (+ 118us) client_negotiation.cc:240] Negotiation successful
0929 15:51:15.791634 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":342,"threads_started":1}
I20260929 15:51:15.827883 17882 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:15.828225 17882 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:15.831784 17882 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc: Sending DeleteTablet for 3 replicas of tablet 46e34af9345e45ef91ddec8ba9924ca5
I20260929 15:51:15.832971 18020 tablet_service.cc:1514] Processing DeleteTablet for tablet 46e34af9345e45ef91ddec8ba9924ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:15 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:43918
I20260929 15:51:15.833192 18298 tablet_service.cc:1514] Processing DeleteTablet for tablet 46e34af9345e45ef91ddec8ba9924ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:15 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:53480
I20260929 15:51:15.835919 18159 tablet_service.cc:1514] Processing DeleteTablet for tablet 46e34af9345e45ef91ddec8ba9924ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:15 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:44628
I20260929 15:51:15.835896 17882 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:51:15.836568 17882 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:51:15.849783 18297 tablet_service.cc:1467] Processing CreateTablet for tablet 9cb4756305ed4d9abe82ddf9c5a9e853 (DEFAULT_TABLE table=test-table [id=0a01b75496e24d5595a3dd3c99435530]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:15.850466 18297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cb4756305ed4d9abe82ddf9c5a9e853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:15.853929 18436 tablet_replica.cc:330] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8: stopping tablet replica
I20260929 15:51:15.855798 18437 tablet_replica.cc:330] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a: stopping tablet replica
I20260929 15:51:15.856735 18019 tablet_service.cc:1467] Processing CreateTablet for tablet 9cb4756305ed4d9abe82ddf9c5a9e853 (DEFAULT_TABLE table=test-table [id=0a01b75496e24d5595a3dd3c99435530]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:15.857309 18019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cb4756305ed4d9abe82ddf9c5a9e853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:15.859651 18387 tablet_bootstrap.cc:492] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a: Bootstrap starting.
I20260929 15:51:15.859853 18158 tablet_service.cc:1467] Processing CreateTablet for tablet 9cb4756305ed4d9abe82ddf9c5a9e853 (DEFAULT_TABLE table=test-table [id=0a01b75496e24d5595a3dd3c99435530]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:15.860387 18158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cb4756305ed4d9abe82ddf9c5a9e853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:15.861393 18387 tablet_bootstrap.cc:654] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:15.865924 18438 tablet_replica.cc:330] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06: stopping tablet replica
I20260929 15:51:15.866562 18438 raft_consensus.cc:2229] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:15.867020 18438 raft_consensus.cc:2258] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:15.868996 18438 ts_tablet_manager.cc:1905] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:15.875982 18388 tablet_bootstrap.cc:492] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06: Bootstrap starting.
I20260929 15:51:15.877092 18438 ts_tablet_manager.cc:1918] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 15:51:15.877368 18438 log.cc:1196] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/46e34af9345e45ef91ddec8ba9924ca5
I20260929 15:51:15.877898 18388 tablet_bootstrap.cc:654] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:15.878237 18438 ts_tablet_manager.cc:1939] T 46e34af9345e45ef91ddec8ba9924ca5 P 769667826c2f47e1b3d103f0a2dd0a06: Deleting consensus metadata
I20260929 15:51:15.880221 17870 catalog_manager.cc:4841] TS 769667826c2f47e1b3d103f0a2dd0a06 (127.15.196.194:44161): tablet 46e34af9345e45ef91ddec8ba9924ca5 (table test-table [id=7577ae33aec44b269737633c25724aed]) successfully deleted
I20260929 15:51:15.883599 18389 tablet_bootstrap.cc:492] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8: Bootstrap starting.
I20260929 15:51:15.884405 18437 raft_consensus.cc:2229] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:15.884800 18436 raft_consensus.cc:2229] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:51:15.884979 18387 tablet_bootstrap.cc:492] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a: No bootstrap required, opened a new log
I20260929 15:51:15.885180 18387 ts_tablet_manager.cc:1397] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.009s
I20260929 15:51:15.885537 18389 tablet_bootstrap.cc:654] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:15.885783 18437 raft_consensus.cc:2258] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:15.886291 18436 raft_consensus.cc:2258] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:15.886029 18387 raft_consensus.cc:348] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.886520 18387 raft_consensus.cc:374] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:15.886675 18387 raft_consensus.cc:729] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d036942aa3349c88175e94143685d9a, State: Initialized, Role: FOLLOWER
I20260929 15:51:15.886936 18387 consensus_queue.cc:260] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.887547 18387 ts_tablet_manager.cc:1428] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260929 15:51:15.888830 18388 tablet_bootstrap.cc:492] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06: No bootstrap required, opened a new log
I20260929 15:51:15.889010 18388 ts_tablet_manager.cc:1397] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.001s
I20260929 15:51:15.889303 18437 ts_tablet_manager.cc:1905] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:15.889770 18388 raft_consensus.cc:348] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.890027 18388 raft_consensus.cc:374] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:15.890112 18388 raft_consensus.cc:729] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 769667826c2f47e1b3d103f0a2dd0a06, State: Initialized, Role: FOLLOWER
I20260929 15:51:15.891878 18436 ts_tablet_manager.cc:1905] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:15.892292 18388 consensus_queue.cc:260] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.895702 18388 ts_tablet_manager.cc:1428] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260929 15:51:15.897861 18437 ts_tablet_manager.cc:1918] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 15:51:15.898088 18437 log.cc:1196] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/46e34af9345e45ef91ddec8ba9924ca5
I20260929 15:51:15.898782 18437 ts_tablet_manager.cc:1939] T 46e34af9345e45ef91ddec8ba9924ca5 P 8d036942aa3349c88175e94143685d9a: Deleting consensus metadata
I20260929 15:51:15.900830 17869 catalog_manager.cc:4841] TS 8d036942aa3349c88175e94143685d9a (127.15.196.195:45567): tablet 46e34af9345e45ef91ddec8ba9924ca5 (table test-table [id=7577ae33aec44b269737633c25724aed]) successfully deleted
I20260929 15:51:15.902896 18436 ts_tablet_manager.cc:1918] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 15:51:15.903088 18436 log.cc:1196] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/46e34af9345e45ef91ddec8ba9924ca5
I20260929 15:51:15.903695 18436 ts_tablet_manager.cc:1939] T 46e34af9345e45ef91ddec8ba9924ca5 P 60d9ce4e736c4bddb9e837ca6428a0f8: Deleting consensus metadata
I20260929 15:51:15.905350 17870 catalog_manager.cc:4841] TS 60d9ce4e736c4bddb9e837ca6428a0f8 (127.15.196.193:34689): tablet 46e34af9345e45ef91ddec8ba9924ca5 (table test-table [id=7577ae33aec44b269737633c25724aed]) successfully deleted
I20260929 15:51:15.907081 18389 tablet_bootstrap.cc:492] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8: No bootstrap required, opened a new log
I20260929 15:51:15.907259 18389 ts_tablet_manager.cc:1397] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.003s
I20260929 15:51:15.908094 18389 raft_consensus.cc:348] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.908354 18389 raft_consensus.cc:374] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:15.908449 18389 raft_consensus.cc:729] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60d9ce4e736c4bddb9e837ca6428a0f8, State: Initialized, Role: FOLLOWER
I20260929 15:51:15.908702 18389 consensus_queue.cc:260] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.909474 18389 ts_tablet_manager.cc:1428] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 15:51:15.988541 18394 raft_consensus.cc:482] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:51:15.988832 18394 raft_consensus.cc:504] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:15.990778 18394 leader_election.cc:288] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60d9ce4e736c4bddb9e837ca6428a0f8 (127.15.196.193:34689), 769667826c2f47e1b3d103f0a2dd0a06 (127.15.196.194:44161)
I20260929 15:51:16.001569 18382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.991254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:54407 (local address 127.15.196.193:34689)
0929 15:51:15.991502 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.991508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:15.992094 (+ 586us) server_negotiation.cc:407] Connection header received
0929 15:51:15.992405 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:15.992411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:15.992489 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:15.992553 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:15.993282 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.994482 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.997424 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.998839 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.998998 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:15.999077 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:16.001090 (+ 2013us) server_negotiation.cc:299] Negotiation successful
0929 15:51:16.001255 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260929 15:51:16.002606 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.991059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34689 (local address 127.15.196.195:54407)
0929 15:51:15.991977 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:15.991999 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:15.992227 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:15.992748 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:15.992754 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:15.992781 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:15.993092 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.993104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:15.994679 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:15.994684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:15.997210 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:15.997227 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:16.000676 (+ 3449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:16.000683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:16.000893 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:16.000934 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 15:51:16.002041 (+ 1107us) client_negotiation.cc:240] Negotiation successful
0929 15:51:16.002221 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"thread_start_us":649,"threads_started":1}
I20260929 15:51:16.005102 18045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb4756305ed4d9abe82ddf9c5a9e853" candidate_uuid: "8d036942aa3349c88175e94143685d9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" is_pre_election: true
I20260929 15:51:16.005441 18045 raft_consensus.cc:2454] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d036942aa3349c88175e94143685d9a in term 0.
I20260929 15:51:16.006117 18252 leader_election.cc:302] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60d9ce4e736c4bddb9e837ca6428a0f8, 8d036942aa3349c88175e94143685d9a; no voters:
I20260929 15:51:16.006718 18394 raft_consensus.cc:2781] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:51:16.006860 18394 raft_consensus.cc:482] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:51:16.006965 18394 raft_consensus.cc:3037] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:16.010455 18394 raft_consensus.cc:504] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:16.011086 18386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.992695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:43645 (local address 127.15.196.194:44161)
0929 15:51:15.992926 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:15.992932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:16.003122 (+ 10190us) server_negotiation.cc:407] Connection header received
0929 15:51:16.003444 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:16.003451 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:16.003537 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:16.003600 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:16.004205 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:16.005460 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:16.008349 (+ 2889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:16.009801 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:16.009948 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:16.010027 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:16.010593 (+ 566us) server_negotiation.cc:299] Negotiation successful
0929 15:51:16.010748 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260929 15:51:16.011245 18394 leader_election.cc:288] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [CANDIDATE]: Term 1 election: Requested vote from peers 60d9ce4e736c4bddb9e837ca6428a0f8 (127.15.196.193:34689), 769667826c2f47e1b3d103f0a2dd0a06 (127.15.196.194:44161)
I20260929 15:51:16.012048 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:15.992181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:44161 (local address 127.15.196.195:43645)
0929 15:51:16.003025 (+ 10844us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:16.003045 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:16.003243 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:16.003817 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:16.003822 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:16.003851 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:16.004028 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:16.004040 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:16.005782 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:16.005788 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:16.008148 (+ 2360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:16.008174 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:16.010195 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:16.010202 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:16.010395 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:16.010430 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 15:51:16.011571 (+ 1141us) client_negotiation.cc:240] Negotiation successful
0929 15:51:16.011724 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10733,"thread_start_us":12159,"threads_started":1}
I20260929 15:51:16.012006 18045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb4756305ed4d9abe82ddf9c5a9e853" candidate_uuid: "8d036942aa3349c88175e94143685d9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8"
I20260929 15:51:16.012248 18045 raft_consensus.cc:3037] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:16.013059 18179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb4756305ed4d9abe82ddf9c5a9e853" candidate_uuid: "8d036942aa3349c88175e94143685d9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "769667826c2f47e1b3d103f0a2dd0a06" is_pre_election: true
I20260929 15:51:16.013331 18179 raft_consensus.cc:2454] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d036942aa3349c88175e94143685d9a in term 0.
I20260929 15:51:16.014110 18179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb4756305ed4d9abe82ddf9c5a9e853" candidate_uuid: "8d036942aa3349c88175e94143685d9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "769667826c2f47e1b3d103f0a2dd0a06"
I20260929 15:51:16.014334 18179 raft_consensus.cc:3037] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:16.015547 18045 raft_consensus.cc:2454] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d036942aa3349c88175e94143685d9a in term 1.
I20260929 15:51:16.016167 18252 leader_election.cc:302] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60d9ce4e736c4bddb9e837ca6428a0f8, 8d036942aa3349c88175e94143685d9a; no voters:
I20260929 15:51:16.016496 18394 raft_consensus.cc:2781] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:16.017571 18179 raft_consensus.cc:2454] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d036942aa3349c88175e94143685d9a in term 1.
I20260929 15:51:16.018272 18394 raft_consensus.cc:686] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [term 1 LEADER]: Becoming Leader. State: Replica: 8d036942aa3349c88175e94143685d9a, State: Running, Role: LEADER
I20260929 15:51:16.018782 18394 consensus_queue.cc:237] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:16.022982 17882 catalog_manager.cc:5495] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d036942aa3349c88175e94143685d9a (127.15.196.195). New cstate: current_term: 1 leader_uuid: "8d036942aa3349c88175e94143685d9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } health_report { overall_health: HEALTHY } } }
I20260929 15:51:16.054075 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:16.045568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53484 (local address 127.15.196.195:45567)
0929 15:51:16.046402 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:16.046409 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:16.046427 (+ 18us) server_negotiation.cc:407] Connection header received
0929 15:51:16.046548 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:16.046554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:16.046680 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:16.046871 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:51:16.047360 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:16.048478 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:16.050490 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:16.051163 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:16.051324 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:16.052000 (+ 676us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:51:16.052883 (+ 883us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:51:16.053038 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:16.053188 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 15:51:16.053514 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695}
I20260929 15:51:16.075834 18386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:16.068458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:59199 (local address 127.15.196.194:44161)
0929 15:51:16.068676 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:16.068683 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:16.069295 (+ 612us) server_negotiation.cc:407] Connection header received
0929 15:51:16.069595 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:16.069603 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:16.069715 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:16.069832 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:51:16.070435 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:16.071673 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:16.073659 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:16.074324 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:16.074469 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:16.074981 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:51:16.075128 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:51:16.075226 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:16.075430 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 15:51:16.075521 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 15:51:16.076594 18448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:16.068584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:44161 (local address 127.15.196.195:59199)
0929 15:51:16.069196 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:16.069215 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:16.069414 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:16.069866 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:16.069871 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:16.069884 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 15:51:16.070229 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:16.070241 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:16.071820 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:16.071826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:16.073504 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:16.073518 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:16.074515 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:16.074522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:16.074702 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:16.074785 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 15:51:16.075241 (+ 456us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 15:51:16.075253 (+ 12us) client_negotiation.cc:769] Sending connection context
0929 15:51:16.076051 (+ 798us) client_negotiation.cc:240] Negotiation successful
0929 15:51:16.076209 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":262,"threads_started":1}
I20260929 15:51:16.087024 18179 raft_consensus.cc:1261] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Refusing update from remote peer 8d036942aa3349c88175e94143685d9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:16.087176 18044 raft_consensus.cc:1261] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Refusing update from remote peer 8d036942aa3349c88175e94143685d9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:16.087847 18444 consensus_queue.cc:1035] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:16.088619 18444 consensus_queue.cc:1035] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:16.101958 18417 mvcc.cc:205] Tried to move back new op lower bound from 7334695223645995008 to 7334695223378399232. Current Snapshot: MvccSnapshot[applied={T|T < 7334695223645995008}]
I20260929 15:51:16.151063 18432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:16.130762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:45567 (local address 127.15.196.194:33357)
0929 15:51:16.130949 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:16.130970 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:16.131085 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:16.131644 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:16.131649 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:16.131662 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 15:51:16.131876 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:16.131890 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:16.135933 (+ 4043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:16.135939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:16.137640 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:16.137655 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:16.147187 (+ 9532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:16.147270 (+ 83us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:16.147531 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:16.147588 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 15:51:16.150439 (+ 2851us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 15:51:16.150458 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 15:51:16.150532 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 15:51:16.150706 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 15:51:16.151847 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:16.130870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:33357 (local address 127.15.196.195:45567)
0929 15:51:16.131048 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:16.131054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:16.131074 (+ 20us) server_negotiation.cc:407] Connection header received
0929 15:51:16.131309 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:16.131316 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:16.131400 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:16.131488 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:51:16.132441 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:16.133697 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:16.146096 (+ 12399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:16.146914 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:16.147041 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:16.150140 (+ 3099us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:51:16.150289 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:51:16.150381 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:16.151440 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0929 15:51:16.151550 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260929 15:51:16.152838 18417 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 15:51:17.797273 17881 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:17.797518 17881 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:17.801956 17881 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc: Sending DeleteTablet for 3 replicas of tablet 9cb4756305ed4d9abe82ddf9c5a9e853
I20260929 15:51:17.802798 18019 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cb4756305ed4d9abe82ddf9c5a9e853 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:17 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:43918
I20260929 15:51:17.802992 18298 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cb4756305ed4d9abe82ddf9c5a9e853 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:17 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:53480
I20260929 15:51:17.803030 18158 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cb4756305ed4d9abe82ddf9c5a9e853 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:17 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:44628
I20260929 15:51:17.815321 18474 tablet_replica.cc:330] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8: stopping tablet replica
I20260929 15:51:17.815500 18476 tablet_replica.cc:330] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06: stopping tablet replica
I20260929 15:51:17.815656 18475 tablet_replica.cc:330] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a: stopping tablet replica
I20260929 15:51:17.815697 18474 raft_consensus.cc:2229] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:17.815901 18476 raft_consensus.cc:2229] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:17.815927 18475 raft_consensus.cc:2229] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:51:17.816018 18474 raft_consensus.cc:2258] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:17.816568 18476 raft_consensus.cc:2258] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:17.816586 18475 raft_consensus.cc:2258] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:17.817078 18474 ts_tablet_manager.cc:1905] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:17.817848 18476 ts_tablet_manager.cc:1905] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:17.817857 18475 ts_tablet_manager.cc:1905] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 29 15:51:17 dist-test-slave-c51v krb5kdc[17838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697077, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260929 15:51:17.852341 18474 ts_tablet_manager.cc:1918] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 15:51:17.852584 18474 log.cc:1196] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/9cb4756305ed4d9abe82ddf9c5a9e853
I20260929 15:51:17.853120 18474 ts_tablet_manager.cc:1939] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 60d9ce4e736c4bddb9e837ca6428a0f8: Deleting consensus metadata
I20260929 15:51:17.857213 17870 catalog_manager.cc:4841] TS 60d9ce4e736c4bddb9e837ca6428a0f8 (127.15.196.193:34689): tablet 9cb4756305ed4d9abe82ddf9c5a9e853 (table test-table [id=0a01b75496e24d5595a3dd3c99435530]) successfully deleted
Sep 29 15:51:17 dist-test-slave-c51v krb5kdc[17838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697077, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:17.872494 18483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:17.858075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50002 (local address 127.15.196.254:34585)
0929 15:51:17.859104 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:17.859108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:17.859126 (+ 18us) server_negotiation.cc:407] Connection header received
0929 15:51:17.859171 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:17.859175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:17.859228 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:17.859457 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:17.860807 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:17.861661 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:17.862908 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:17.863415 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:17.863518 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:17.867547 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:17.867552 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:17.867556 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:17.867592 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:17.870131 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:17.870839 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:17.870842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:17.870845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:17.870967 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:17.871271 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:17.871274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:17.871276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:17.871608 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:17.871790 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:17.872066 (+ 276us) server_negotiation.cc:299] Negotiation successful
0929 15:51:17.872234 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":921,"thread_start_us":779,"threads_started":1}
I20260929 15:51:17.876251 17881 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:51:17.876932 17881 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:51:17.883682 18019 tablet_service.cc:1467] Processing CreateTablet for tablet 33c6c426294643eead0209f88201174b (DEFAULT_TABLE table=test-table [id=7a45184d9837430eb435d2cc41785776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:17.884068 18298 tablet_service.cc:1467] Processing CreateTablet for tablet 33c6c426294643eead0209f88201174b (DEFAULT_TABLE table=test-table [id=7a45184d9837430eb435d2cc41785776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:17.884192 18019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33c6c426294643eead0209f88201174b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:17.884490 18158 tablet_service.cc:1467] Processing CreateTablet for tablet 33c6c426294643eead0209f88201174b (DEFAULT_TABLE table=test-table [id=7a45184d9837430eb435d2cc41785776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:17.889339 18485 tablet_bootstrap.cc:492] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8: Bootstrap starting.
I20260929 15:51:17.891124 18485 tablet_bootstrap.cc:654] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:17.893312 18485 tablet_bootstrap.cc:492] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8: No bootstrap required, opened a new log
I20260929 15:51:17.893483 18485 ts_tablet_manager.cc:1397] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260929 15:51:17.894302 18485 raft_consensus.cc:348] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:17.894515 18485 raft_consensus.cc:374] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:17.894593 18485 raft_consensus.cc:729] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60d9ce4e736c4bddb9e837ca6428a0f8, State: Initialized, Role: FOLLOWER
I20260929 15:51:17.894762 18485 consensus_queue.cc:260] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:17.895517 18485 ts_tablet_manager.cc:1428] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 15:51:17.917491 18158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33c6c426294643eead0209f88201174b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:17.917538 18476 ts_tablet_manager.cc:1918] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 15:51:17.917551 18475 ts_tablet_manager.cc:1918] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 15:51:17.917662 18476 log.cc:1196] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/9cb4756305ed4d9abe82ddf9c5a9e853
I20260929 15:51:17.917712 18475 log.cc:1196] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/9cb4756305ed4d9abe82ddf9c5a9e853
I20260929 15:51:17.918156 18476 ts_tablet_manager.cc:1939] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 769667826c2f47e1b3d103f0a2dd0a06: Deleting consensus metadata
I20260929 15:51:17.918184 18475 ts_tablet_manager.cc:1939] T 9cb4756305ed4d9abe82ddf9c5a9e853 P 8d036942aa3349c88175e94143685d9a: Deleting consensus metadata
I20260929 15:51:17.918514 18298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33c6c426294643eead0209f88201174b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:17.923506 18488 tablet_bootstrap.cc:492] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06: Bootstrap starting.
I20260929 15:51:17.923524 17869 catalog_manager.cc:4841] TS 8d036942aa3349c88175e94143685d9a (127.15.196.195:45567): tablet 9cb4756305ed4d9abe82ddf9c5a9e853 (table test-table [id=0a01b75496e24d5595a3dd3c99435530]) successfully deleted
I20260929 15:51:17.923983 18487 raft_consensus.cc:482] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:51:17.924201 18487 raft_consensus.cc:504] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:17.925010 18487 leader_election.cc:288] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 769667826c2f47e1b3d103f0a2dd0a06 (127.15.196.194:44161), 8d036942aa3349c88175e94143685d9a (127.15.196.195:45567)
I20260929 15:51:17.925364 18488 tablet_bootstrap.cc:654] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:17.926652 18489 tablet_bootstrap.cc:492] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a: Bootstrap starting.
I20260929 15:51:17.927802 18179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33c6c426294643eead0209f88201174b" candidate_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "769667826c2f47e1b3d103f0a2dd0a06" is_pre_election: true
W20260929 15:51:17.928870 17973 leader_election.cc:341] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 769667826c2f47e1b3d103f0a2dd0a06 (127.15.196.194:44161): Illegal state: must be running to vote when last-logged opid is not known
I20260929 15:51:17.929277 18489 tablet_bootstrap.cc:654] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:17.929574 18322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33c6c426294643eead0209f88201174b" candidate_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d036942aa3349c88175e94143685d9a" is_pre_election: true
I20260929 15:51:17.929651 17870 catalog_manager.cc:4841] TS 769667826c2f47e1b3d103f0a2dd0a06 (127.15.196.194:44161): tablet 9cb4756305ed4d9abe82ddf9c5a9e853 (table test-table [id=0a01b75496e24d5595a3dd3c99435530]) successfully deleted
W20260929 15:51:17.930385 17972 leader_election.cc:341] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8d036942aa3349c88175e94143685d9a (127.15.196.195:45567): Illegal state: must be running to vote when last-logged opid is not known
I20260929 15:51:17.930570 17972 leader_election.cc:302] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60d9ce4e736c4bddb9e837ca6428a0f8; no voters: 769667826c2f47e1b3d103f0a2dd0a06, 8d036942aa3349c88175e94143685d9a
I20260929 15:51:17.930908 18487 raft_consensus.cc:2726] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260929 15:51:17.931389 18488 tablet_bootstrap.cc:492] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06: No bootstrap required, opened a new log
I20260929 15:51:17.931553 18488 ts_tablet_manager.cc:1397] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260929 15:51:17.931783 18489 tablet_bootstrap.cc:492] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a: No bootstrap required, opened a new log
I20260929 15:51:17.931942 18489 ts_tablet_manager.cc:1397] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 15:51:17.932317 18488 raft_consensus.cc:348] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:17.932502 18488 raft_consensus.cc:374] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:17.932613 18488 raft_consensus.cc:729] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 769667826c2f47e1b3d103f0a2dd0a06, State: Initialized, Role: FOLLOWER
I20260929 15:51:17.932650 18489 raft_consensus.cc:348] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:17.932951 18489 raft_consensus.cc:374] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:17.932889 18488 consensus_queue.cc:260] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:17.933048 18489 raft_consensus.cc:729] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d036942aa3349c88175e94143685d9a, State: Initialized, Role: FOLLOWER
I20260929 15:51:17.933274 18489 consensus_queue.cc:260] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:17.933606 18488 ts_tablet_manager.cc:1428] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 15:51:17.933904 18489 ts_tablet_manager.cc:1428] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 15:51:18.082720 18453 raft_consensus.cc:482] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:51:18.082967 18453 raft_consensus.cc:504] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:18.083690 18453 leader_election.cc:288] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60d9ce4e736c4bddb9e837ca6428a0f8 (127.15.196.193:34689), 8d036942aa3349c88175e94143685d9a (127.15.196.195:45567)
I20260929 15:51:18.084336 18322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33c6c426294643eead0209f88201174b" candidate_uuid: "769667826c2f47e1b3d103f0a2dd0a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d036942aa3349c88175e94143685d9a" is_pre_election: true
I20260929 15:51:18.084384 18043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33c6c426294643eead0209f88201174b" candidate_uuid: "769667826c2f47e1b3d103f0a2dd0a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" is_pre_election: true
I20260929 15:51:18.084614 18322 raft_consensus.cc:2454] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 769667826c2f47e1b3d103f0a2dd0a06 in term 0.
I20260929 15:51:18.084640 18043 raft_consensus.cc:2454] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 769667826c2f47e1b3d103f0a2dd0a06 in term 0.
I20260929 15:51:18.085130 18113 leader_election.cc:302] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60d9ce4e736c4bddb9e837ca6428a0f8, 769667826c2f47e1b3d103f0a2dd0a06; no voters:
I20260929 15:51:18.085469 18453 raft_consensus.cc:2781] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:51:18.085564 18453 raft_consensus.cc:482] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:51:18.085711 18453 raft_consensus.cc:3037] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:18.129837 18453 raft_consensus.cc:504] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:18.130750 18453 leader_election.cc:288] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [CANDIDATE]: Term 1 election: Requested vote from peers 60d9ce4e736c4bddb9e837ca6428a0f8 (127.15.196.193:34689), 8d036942aa3349c88175e94143685d9a (127.15.196.195:45567)
I20260929 15:51:18.131191 18043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33c6c426294643eead0209f88201174b" candidate_uuid: "769667826c2f47e1b3d103f0a2dd0a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8"
I20260929 15:51:18.131482 18043 raft_consensus.cc:3037] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:18.131507 18322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33c6c426294643eead0209f88201174b" candidate_uuid: "769667826c2f47e1b3d103f0a2dd0a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d036942aa3349c88175e94143685d9a"
I20260929 15:51:18.131745 18322 raft_consensus.cc:3037] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:18.134702 18322 raft_consensus.cc:2454] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 769667826c2f47e1b3d103f0a2dd0a06 in term 1.
I20260929 15:51:18.134727 18043 raft_consensus.cc:2454] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 769667826c2f47e1b3d103f0a2dd0a06 in term 1.
I20260929 15:51:18.135264 18113 leader_election.cc:302] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60d9ce4e736c4bddb9e837ca6428a0f8, 769667826c2f47e1b3d103f0a2dd0a06; no voters:
I20260929 15:51:18.135679 18453 raft_consensus.cc:2781] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:18.135919 18453 raft_consensus.cc:686] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 LEADER]: Becoming Leader. State: Replica: 769667826c2f47e1b3d103f0a2dd0a06, State: Running, Role: LEADER
I20260929 15:51:18.136236 18453 consensus_queue.cc:237] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:18.139164 17882 catalog_manager.cc:5495] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 769667826c2f47e1b3d103f0a2dd0a06 (127.15.196.194). New cstate: current_term: 1 leader_uuid: "769667826c2f47e1b3d103f0a2dd0a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } health_report { overall_health: UNKNOWN } } }
I20260929 15:51:18.201785 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:18.195808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44642 (local address 127.15.196.194:44161)
0929 15:51:18.196392 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:18.196398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:18.196419 (+ 21us) server_negotiation.cc:407] Connection header received
0929 15:51:18.196481 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:18.196486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:18.196563 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:18.196661 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:51:18.197007 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:18.198172 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:18.199747 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:18.200438 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:18.200553 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:18.200940 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:51:18.201073 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:51:18.201168 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:18.201365 (+ 197us) server_negotiation.cc:299] Negotiation successful
0929 15:51:18.201443 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":311,"threads_started":1}
I20260929 15:51:18.216406 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:18.211037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:54787 (local address 127.15.196.194:44161)
0929 15:51:18.211232 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:18.211241 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:18.211663 (+ 422us) server_negotiation.cc:407] Connection header received
0929 15:51:18.211813 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:18.211818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:18.211893 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:18.211983 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:51:18.212411 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:18.213458 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:18.214734 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:18.215383 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:18.215487 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:18.215815 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:51:18.215905 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:51:18.215968 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:18.216087 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 15:51:18.216147 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 15:51:18.216500 18497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:18.210758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:44161 (local address 127.15.196.194:54787)
0929 15:51:18.211574 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:18.211591 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:18.211702 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:18.212031 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:18.212034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:18.212048 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 15:51:18.212278 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:18.212285 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:18.213572 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:18.213575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:18.214603 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:18.214613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:18.215537 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:18.215541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:18.215656 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:18.215692 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 15:51:18.215986 (+ 294us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 15:51:18.215994 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 15:51:18.216053 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 15:51:18.216151 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":3,"thread_start_us":565,"threads_started":1}
I20260929 15:51:18.224181 18322 raft_consensus.cc:1261] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a [term 1 FOLLOWER]: Refusing update from remote peer 769667826c2f47e1b3d103f0a2dd0a06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:18.224289 18043 raft_consensus.cc:1261] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Refusing update from remote peer 769667826c2f47e1b3d103f0a2dd0a06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:18.224761 18498 consensus_queue.cc:1035] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:18.225075 18453 consensus_queue.cc:1035] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:18.279341 17881 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:18.279663 17881 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:18.283142 17881 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc: Sending DeleteTablet for 3 replicas of tablet 33c6c426294643eead0209f88201174b
I20260929 15:51:18.284006 18019 tablet_service.cc:1514] Processing DeleteTablet for tablet 33c6c426294643eead0209f88201174b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:18 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:43918
I20260929 15:51:18.284240 18298 tablet_service.cc:1514] Processing DeleteTablet for tablet 33c6c426294643eead0209f88201174b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:18 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:53480
I20260929 15:51:18.284300 18474 tablet_replica.cc:330] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8: stopping tablet replica
I20260929 15:51:18.284540 18475 tablet_replica.cc:330] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a: stopping tablet replica
I20260929 15:51:18.284528 18474 raft_consensus.cc:2229] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:18.284727 18474 raft_consensus.cc:2258] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:18.284729 18475 raft_consensus.cc:2229] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:18.284901 18475 raft_consensus.cc:2258] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:18.285204 18158 tablet_service.cc:1514] Processing DeleteTablet for tablet 33c6c426294643eead0209f88201174b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:18 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:44628
I20260929 15:51:18.285502 18476 tablet_replica.cc:330] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06: stopping tablet replica
I20260929 15:51:18.285694 18476 raft_consensus.cc:2229] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:51:18.285888 17881 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 15:51:18.286130 18474 ts_tablet_manager.cc:1905] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:18.286217 18476 raft_consensus.cc:2258] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260929 15:51:18.286525 17881 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:51:18.287371 18475 ts_tablet_manager.cc:1905] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:18.287369 18476 ts_tablet_manager.cc:1905] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:18.295487 18019 tablet_service.cc:1467] Processing CreateTablet for tablet eea24ee704bc407caf83d7d9d646507f (DEFAULT_TABLE table=test-table [id=a8720537af744142b35d51c2d624dcf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:18.295804 18298 tablet_service.cc:1467] Processing CreateTablet for tablet eea24ee704bc407caf83d7d9d646507f (DEFAULT_TABLE table=test-table [id=a8720537af744142b35d51c2d624dcf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:18.296415 18158 tablet_service.cc:1467] Processing CreateTablet for tablet eea24ee704bc407caf83d7d9d646507f (DEFAULT_TABLE table=test-table [id=a8720537af744142b35d51c2d624dcf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:18.335222 18019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eea24ee704bc407caf83d7d9d646507f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:18.335963 18298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eea24ee704bc407caf83d7d9d646507f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:18.341791 18158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eea24ee704bc407caf83d7d9d646507f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:18.344993 18485 tablet_bootstrap.cc:492] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8: Bootstrap starting.
I20260929 15:51:18.346443 18489 tablet_bootstrap.cc:492] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a: Bootstrap starting.
I20260929 15:51:18.346686 18485 tablet_bootstrap.cc:654] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:18.348171 18489 tablet_bootstrap.cc:654] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:18.354949 18488 tablet_bootstrap.cc:492] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06: Bootstrap starting.
I20260929 15:51:18.357249 18488 tablet_bootstrap.cc:654] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:18.357673 18485 tablet_bootstrap.cc:492] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8: No bootstrap required, opened a new log
I20260929 15:51:18.357844 18485 ts_tablet_manager.cc:1397] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260929 15:51:18.358597 18485 raft_consensus.cc:348] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:18.358923 18485 raft_consensus.cc:374] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:18.359040 18485 raft_consensus.cc:729] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60d9ce4e736c4bddb9e837ca6428a0f8, State: Initialized, Role: FOLLOWER
I20260929 15:51:18.359414 18485 consensus_queue.cc:260] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:18.360888 18485 ts_tablet_manager.cc:1428] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:18.365554 18489 tablet_bootstrap.cc:492] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a: No bootstrap required, opened a new log
I20260929 15:51:18.365754 18489 ts_tablet_manager.cc:1397] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s
I20260929 15:51:18.366181 18488 tablet_bootstrap.cc:492] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06: No bootstrap required, opened a new log
I20260929 15:51:18.366354 18488 ts_tablet_manager.cc:1397] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260929 15:51:18.366600 18489 raft_consensus.cc:348] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:18.366868 18489 raft_consensus.cc:374] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:18.366953 18489 raft_consensus.cc:729] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d036942aa3349c88175e94143685d9a, State: Initialized, Role: FOLLOWER
I20260929 15:51:18.367197 18489 consensus_queue.cc:260] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:18.367179 18488 raft_consensus.cc:348] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:18.367471 18488 raft_consensus.cc:374] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:18.367551 18488 raft_consensus.cc:729] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 769667826c2f47e1b3d103f0a2dd0a06, State: Initialized, Role: FOLLOWER
I20260929 15:51:18.367791 18488 consensus_queue.cc:260] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:18.368351 18488 ts_tablet_manager.cc:1428] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 15:51:18.367996 18489 ts_tablet_manager.cc:1428] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 15:51:18.441526 18474 ts_tablet_manager.cc:1918] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 15:51:18.441531 18476 ts_tablet_manager.cc:1918] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 15:51:18.441745 18476 log.cc:1196] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/33c6c426294643eead0209f88201174b
I20260929 15:51:18.441759 18474 log.cc:1196] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/33c6c426294643eead0209f88201174b
I20260929 15:51:18.442226 18474 ts_tablet_manager.cc:1939] T 33c6c426294643eead0209f88201174b P 60d9ce4e736c4bddb9e837ca6428a0f8: Deleting consensus metadata
I20260929 15:51:18.442255 18476 ts_tablet_manager.cc:1939] T 33c6c426294643eead0209f88201174b P 769667826c2f47e1b3d103f0a2dd0a06: Deleting consensus metadata
I20260929 15:51:18.443207 18475 ts_tablet_manager.cc:1918] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 15:51:18.443754 17870 catalog_manager.cc:4841] TS 769667826c2f47e1b3d103f0a2dd0a06 (127.15.196.194:44161): tablet 33c6c426294643eead0209f88201174b (table test-table [id=7a45184d9837430eb435d2cc41785776]) successfully deleted
I20260929 15:51:18.444144 18475 log.cc:1196] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/33c6c426294643eead0209f88201174b
I20260929 15:51:18.444391 17870 catalog_manager.cc:4841] TS 60d9ce4e736c4bddb9e837ca6428a0f8 (127.15.196.193:34689): tablet 33c6c426294643eead0209f88201174b (table test-table [id=7a45184d9837430eb435d2cc41785776]) successfully deleted
I20260929 15:51:18.444674 18475 ts_tablet_manager.cc:1939] T 33c6c426294643eead0209f88201174b P 8d036942aa3349c88175e94143685d9a: Deleting consensus metadata
I20260929 15:51:18.493476 17869 catalog_manager.cc:4841] TS 8d036942aa3349c88175e94143685d9a (127.15.196.195:45567): tablet 33c6c426294643eead0209f88201174b (table test-table [id=7a45184d9837430eb435d2cc41785776]) successfully deleted
I20260929 15:51:18.613385 18451 raft_consensus.cc:482] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:51:18.613658 18451 raft_consensus.cc:504] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:18.614519 18451 leader_election.cc:288] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60d9ce4e736c4bddb9e837ca6428a0f8 (127.15.196.193:34689), 769667826c2f47e1b3d103f0a2dd0a06 (127.15.196.194:44161)
I20260929 15:51:18.615155 18043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eea24ee704bc407caf83d7d9d646507f" candidate_uuid: "8d036942aa3349c88175e94143685d9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" is_pre_election: true
I20260929 15:51:18.615314 18179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eea24ee704bc407caf83d7d9d646507f" candidate_uuid: "8d036942aa3349c88175e94143685d9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "769667826c2f47e1b3d103f0a2dd0a06" is_pre_election: true
I20260929 15:51:18.615456 18043 raft_consensus.cc:2454] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d036942aa3349c88175e94143685d9a in term 0.
I20260929 15:51:18.615692 18179 raft_consensus.cc:2454] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d036942aa3349c88175e94143685d9a in term 0.
I20260929 15:51:18.615932 18252 leader_election.cc:302] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60d9ce4e736c4bddb9e837ca6428a0f8, 8d036942aa3349c88175e94143685d9a; no voters:
I20260929 15:51:18.616250 18451 raft_consensus.cc:2781] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:51:18.616402 18451 raft_consensus.cc:482] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:51:18.616523 18451 raft_consensus.cc:3037] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:18.642656 18451 raft_consensus.cc:504] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:18.643580 18451 leader_election.cc:288] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [CANDIDATE]: Term 1 election: Requested vote from peers 60d9ce4e736c4bddb9e837ca6428a0f8 (127.15.196.193:34689), 769667826c2f47e1b3d103f0a2dd0a06 (127.15.196.194:44161)
I20260929 15:51:18.644290 18043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eea24ee704bc407caf83d7d9d646507f" candidate_uuid: "8d036942aa3349c88175e94143685d9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8"
I20260929 15:51:18.644295 18179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eea24ee704bc407caf83d7d9d646507f" candidate_uuid: "8d036942aa3349c88175e94143685d9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "769667826c2f47e1b3d103f0a2dd0a06"
I20260929 15:51:18.644531 18043 raft_consensus.cc:3037] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:18.644558 18179 raft_consensus.cc:3037] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:18.745621 18179 raft_consensus.cc:2454] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d036942aa3349c88175e94143685d9a in term 1.
I20260929 15:51:18.745649 18043 raft_consensus.cc:2454] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d036942aa3349c88175e94143685d9a in term 1.
I20260929 15:51:18.746331 18252 leader_election.cc:302] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60d9ce4e736c4bddb9e837ca6428a0f8, 8d036942aa3349c88175e94143685d9a; no voters:
I20260929 15:51:18.746735 18451 raft_consensus.cc:2781] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:18.747236 18451 raft_consensus.cc:686] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [term 1 LEADER]: Becoming Leader. State: Replica: 8d036942aa3349c88175e94143685d9a, State: Running, Role: LEADER
I20260929 15:51:18.747587 18451 consensus_queue.cc:237] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } }
I20260929 15:51:18.750676 17881 catalog_manager.cc:5495] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d036942aa3349c88175e94143685d9a (127.15.196.195). New cstate: current_term: 1 leader_uuid: "8d036942aa3349c88175e94143685d9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d036942aa3349c88175e94143685d9a" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 45567 } health_report { overall_health: HEALTHY } } }
I20260929 15:51:18.785348 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:18.780177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53486 (local address 127.15.196.195:45567)
0929 15:51:18.780859 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:18.780864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:18.780880 (+ 16us) server_negotiation.cc:407] Connection header received
0929 15:51:18.780931 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:18.780935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:18.780993 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:18.781075 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:51:18.781495 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:18.782267 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:18.783725 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:18.784123 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:18.784217 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:18.784659 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:51:18.784761 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:51:18.784840 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:18.785021 (+ 181us) server_negotiation.cc:299] Negotiation successful
0929 15:51:18.785082 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":466,"threads_started":1}
I20260929 15:51:18.794611 18043 raft_consensus.cc:1261] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Refusing update from remote peer 8d036942aa3349c88175e94143685d9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:18.794726 18179 raft_consensus.cc:1261] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Refusing update from remote peer 8d036942aa3349c88175e94143685d9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:18.795219 18507 consensus_queue.cc:1035] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "60d9ce4e736c4bddb9e837ca6428a0f8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:18.795595 18451 consensus_queue.cc:1035] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "769667826c2f47e1b3d103f0a2dd0a06" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 15:51:18.816313 18512 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 15:51:20.414407 17882 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:20.414630 17882 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:20.419023 17882 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a5a0767498364440b596e38e61fc7ecc: Sending DeleteTablet for 3 replicas of tablet eea24ee704bc407caf83d7d9d646507f
I20260929 15:51:20.419953 18019 tablet_service.cc:1514] Processing DeleteTablet for tablet eea24ee704bc407caf83d7d9d646507f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:20 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:43918
I20260929 15:51:20.420008 18298 tablet_service.cc:1514] Processing DeleteTablet for tablet eea24ee704bc407caf83d7d9d646507f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:20 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:53480
I20260929 15:51:20.420061 18158 tablet_service.cc:1514] Processing DeleteTablet for tablet eea24ee704bc407caf83d7d9d646507f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:20 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:44628
I20260929 15:51:20.420737 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 17954
I20260929 15:51:20.432673 18547 tablet_replica.cc:330] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8: stopping tablet replica
I20260929 15:51:20.433048 18547 raft_consensus.cc:2229] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:20.433571 18547 raft_consensus.cc:2258] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:20.435061 18547 ts_tablet_manager.cc:1905] T eea24ee704bc407caf83d7d9d646507f P 60d9ce4e736c4bddb9e837ca6428a0f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:20.436345 18549 tablet_replica.cc:330] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06: stopping tablet replica
I20260929 15:51:20.436596 18549 raft_consensus.cc:2229] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:20.436862 18549 raft_consensus.cc:2258] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:20.437000 18548 tablet_replica.cc:330] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a: stopping tablet replica
I20260929 15:51:20.437228 18548 raft_consensus.cc:2229] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:51:20.437649 18548 raft_consensus.cc:2258] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:20.438643 18548 ts_tablet_manager.cc:1905] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:20.438756 18549 ts_tablet_manager.cc:1905] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 29 15:51:20 dist-test-slave-c51v krb5kdc[17838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697077, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:51:20.458499 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:20.424169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39976 (local address 127.15.196.193:34689)
0929 15:51:20.435652 (+ 11483us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:20.435661 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:20.435693 (+ 32us) server_negotiation.cc:407] Connection header received
0929 15:51:20.435770 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:20.435780 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:20.435895 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:20.436794 (+ 899us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:20.438010 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:20.439224 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:20.441077 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:20.441727 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:20.441844 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:20.452755 (+ 10911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:20.452772 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:20.452789 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:20.452852 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:20.456051 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:20.456651 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:20.456658 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:20.456660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:20.456799 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:20.457114 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:20.457117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:20.457119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:20.457509 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:20.457756 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:20.457985 (+ 229us) server_negotiation.cc:299] Negotiation successful
0929 15:51:20.458147 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11267,"thread_start_us":11858,"threads_started":1}
W20260929 15:51:20.459179 18085 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39976
W20260929 15:51:20.459879 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 17954: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:51:20.460158 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 17954
I20260929 15:51:20.506655 18548 ts_tablet_manager.cc:1918] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 15:51:20.506893 18548 log.cc:1196] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/eea24ee704bc407caf83d7d9d646507f
I20260929 15:51:20.507366 18548 ts_tablet_manager.cc:1939] T eea24ee704bc407caf83d7d9d646507f P 8d036942aa3349c88175e94143685d9a: Deleting consensus metadata
I20260929 15:51:20.515446 18549 ts_tablet_manager.cc:1918] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 15:51:20.515589 18549 log.cc:1196] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/eea24ee704bc407caf83d7d9d646507f
I20260929 15:51:20.516021 18549 ts_tablet_manager.cc:1939] T eea24ee704bc407caf83d7d9d646507f P 769667826c2f47e1b3d103f0a2dd0a06: Deleting consensus metadata
W20260929 15:51:20.528633 17870 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.196.193:34689 (error 108)
I20260929 15:51:20.528915 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 18094
I20260929 15:51:20.530930 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:20.529971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34689 (local address 127.0.0.1:39980)
0929 15:51:20.530517 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:20.530666 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:34689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":296,"threads_started":1}
W20260929 15:51:20.531176 17870 catalog_manager.cc:4584] TS 60d9ce4e736c4bddb9e837ca6428a0f8 (127.15.196.193:34689): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eea24ee704bc407caf83d7d9d646507f: Network error: Client connection negotiation failed: client connection to 127.15.196.193:34689: connect: Connection refused (error 111)
Sep 29 15:51:20 dist-test-slave-c51v krb5kdc[17838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697077, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:51:20.544206 18553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:20.529804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52918 (local address 127.15.196.194:44161)
0929 15:51:20.530372 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:20.530378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:20.530399 (+ 21us) server_negotiation.cc:407] Connection header received
0929 15:51:20.530465 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:20.530470 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:20.530552 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:20.530675 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:20.532227 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:20.533044 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:20.534322 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:20.534732 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:20.534822 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:20.538833 (+ 4011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:20.538849 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:20.538864 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:20.538903 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:20.541588 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:20.542258 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:20.542266 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:20.542268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:20.542419 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:20.542716 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:20.542719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:20.542721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:20.543150 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:20.543458 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:20.543766 (+ 308us) server_negotiation.cc:299] Negotiation successful
0929 15:51:20.543943 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":243,"threads_started":1}
W20260929 15:51:20.544867 18224 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52918
W20260929 15:51:20.545444 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 18094: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:51:20.545575 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 18094
I20260929 15:51:20.584888 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:20.584338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34689 (local address 127.0.0.1:39992)
0929 15:51:20.584574 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:20.584686 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:34689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260929 15:51:20.585250 17869 catalog_manager.cc:4841] TS 8d036942aa3349c88175e94143685d9a (127.15.196.195:45567): tablet eea24ee704bc407caf83d7d9d646507f (table test-table [id=a8720537af744142b35d51c2d624dcf3]) successfully deleted
I20260929 15:51:20.586055 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:20.585703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34689 (local address 127.0.0.1:40000)
0929 15:51:20.585837 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:20.585927 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:34689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260929 15:51:20.606990 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 18233
I20260929 15:51:20.608109 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:20.607658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:44161 (local address 127.0.0.1:52934)
0929 15:51:20.607831 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:20.607936 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.194:44161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
Sep 29 15:51:20 dist-test-slave-c51v krb5kdc[17838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697077, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:51:20.622821 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:20.607797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37226 (local address 127.15.196.195:45567)
0929 15:51:20.608422 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:20.608429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:20.608455 (+ 26us) server_negotiation.cc:407] Connection header received
0929 15:51:20.608510 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:20.608514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:20.608571 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:20.608660 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:20.609977 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:20.610824 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:20.612088 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:20.612493 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:20.612592 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:20.617492 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:20.617508 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:20.617525 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:20.617563 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:20.620519 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:20.621127 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:20.621133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:20.621135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:20.621265 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:20.621589 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:20.621592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:20.621594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:20.621931 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:20.622144 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:20.622388 (+ 244us) server_negotiation.cc:299] Negotiation successful
0929 15:51:20.622554 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":301,"threads_started":1}
W20260929 15:51:20.623588 18363 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37226
W20260929 15:51:20.624193 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 18233: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:51:20.624394 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 18233
I20260929 15:51:20.640992 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:20.640282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:44161 (local address 127.0.0.1:52938)
0929 15:51:20.640555 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:20.640682 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.194:44161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260929 15:51:20.642177 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:20.641690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:44161 (local address 127.0.0.1:52948)
0929 15:51:20.641910 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:20.642011 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.194:44161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 15:51:20.646384 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:20.645912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34689 (local address 127.0.0.1:40014)
0929 15:51:20.646101 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:20.646196 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:34689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 15:51:20.647095 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 17851
I20260929 15:51:20.647625 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:20.647313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34689 (local address 127.0.0.1:40018)
0929 15:51:20.647432 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:20.647502 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:34689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260929 15:51:20.648283 17923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:20.766145 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 17851
2026-09-29T15:51:20Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9031 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:51:22 dist-test-slave-c51v krb5kdc[18560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:51:22 dist-test-slave-c51v krb5kdc[18560](info): set up 2 sockets
Sep 29 15:51:22 dist-test-slave-c51v krb5kdc[18560](info): commencing operation
krb5kdc: starting...
W20260929 15:51:23.681713 16147 mini_kdc.cc:121] Time spent starting KDC: real 2.870s user 0.007s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:51:23 dist-test-slave-c51v krb5kdc[18560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:51:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:51:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:51:23.870532 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:34303
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:43437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:34303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:24.203696 18573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:24.204003 18573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:24.204156 18573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:24.211527 18573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:24.211645 18573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:24.211755 18573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:24.211855 18573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:24.211951 18573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:51:24.222136 18573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:43437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:34303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:34303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:24.223551 18573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:24.224897 18573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:24.237254 18578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:24.237218 18579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:24.237289 18581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:24.238516 18573 server_base.cc:1034] running on GCE node
I20260929 15:51:24.239499 18573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:24.241034 18573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:24.242285 18573 hybrid_clock.cc:648] HybridClock initialized: now 1790697084242235 us; error 30 us; skew 500 ppm
I20260929 15:51:24.242787 18573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:24 dist-test-slave-c51v krb5kdc[18560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697084, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:24.246236 18573 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:51:24.247771 18573 webserver.cc:460] Webserver started at http://127.15.196.254:41295/ using document root <none> and password file <none>
I20260929 15:51:24.248440 18573 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:24.248539 18573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:24.248795 18573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:24.251853 18573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "7e33db87b7da424f9cf494b81b0a145b"
format_stamp: "Formatted at 2026-09-29 15:51:24 on dist-test-slave-c51v"
server_key: "f9276b41644b63cdc2dc97bcebe8097e"
server_key_iv: "e4ce30fbac2db10daf0eb0be67592da5"
server_key_version: "encryptionkey@0"
I20260929 15:51:24.252574 18573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "7e33db87b7da424f9cf494b81b0a145b"
format_stamp: "Formatted at 2026-09-29 15:51:24 on dist-test-slave-c51v"
server_key: "f9276b41644b63cdc2dc97bcebe8097e"
server_key_iv: "e4ce30fbac2db10daf0eb0be67592da5"
server_key_version: "encryptionkey@0"
I20260929 15:51:24.258069 18573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 15:51:24.261610 18588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:24.263053 18573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 15:51:24.263250 18573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "7e33db87b7da424f9cf494b81b0a145b"
format_stamp: "Formatted at 2026-09-29 15:51:24 on dist-test-slave-c51v"
server_key: "f9276b41644b63cdc2dc97bcebe8097e"
server_key_iv: "e4ce30fbac2db10daf0eb0be67592da5"
server_key_version: "encryptionkey@0"
I20260929 15:51:24.263531 18573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:24.296367 18573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:24.300889 18573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:24.301211 18573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:24.319686 18573 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:34303
I20260929 15:51:24.319705 18639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:34303 every 8 connection(s)
I20260929 15:51:24.321127 18573 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:51:24.325153 18640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:24.329686 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 18573
I20260929 15:51:24.329945 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:51:24.330914 16147 external_mini_cluster.cc:1428] Setting key d30d416b4e6149e7e8f6bd96c1c22354
I20260929 15:51:24.337445 18640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b: Bootstrap starting.
I20260929 15:51:24.340669 18640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:24.341671 18640 log.cc:824] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:24.345257 18640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b: No bootstrap required, opened a new log
Sep 29 15:51:24 dist-test-slave-c51v krb5kdc[18560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:24.356055 18640 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e33db87b7da424f9cf494b81b0a145b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34303 } }
I20260929 15:51:24.356501 18640 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:24.356597 18643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:24.333159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46620 (local address 127.15.196.254:34303)
0929 15:51:24.333988 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:24.334001 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:24.334045 (+ 44us) server_negotiation.cc:407] Connection header received
0929 15:51:24.336357 (+ 2312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:24.336383 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:24.336809 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:24.337232 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:24.338885 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:24.340326 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:24.341761 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:24.342931 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:24.343144 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:24.347884 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:24.347895 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:24.347915 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:24.347971 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:24.352028 (+ 4057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:24.352703 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:24.352712 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:24.352722 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:24.352955 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:24.353289 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:24.353300 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:24.353303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:24.354081 (+ 778us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:24.354397 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:24.354708 (+ 311us) server_negotiation.cc:299] Negotiation successful
0929 15:51:24.355176 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":356,"threads_started":1}
I20260929 15:51:24.356624 18640 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e33db87b7da424f9cf494b81b0a145b, State: Initialized, Role: FOLLOWER
I20260929 15:51:24.357285 18640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e33db87b7da424f9cf494b81b0a145b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34303 } }
I20260929 15:51:24.357620 18640 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:51:24.357792 18640 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:51:24.357973 18640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:24.360599 18640 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e33db87b7da424f9cf494b81b0a145b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34303 } }
I20260929 15:51:24.361094 18640 leader_election.cc:302] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e33db87b7da424f9cf494b81b0a145b; no voters:
I20260929 15:51:24.361706 18640 leader_election.cc:288] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:51:24.361864 18645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:24.362903 18645 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [term 1 LEADER]: Becoming Leader. State: Replica: 7e33db87b7da424f9cf494b81b0a145b, State: Running, Role: LEADER
I20260929 15:51:24.363515 18645 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e33db87b7da424f9cf494b81b0a145b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34303 } }
I20260929 15:51:24.363974 18640 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:24.366508 18647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e33db87b7da424f9cf494b81b0a145b. Latest consensus state: current_term: 1 leader_uuid: "7e33db87b7da424f9cf494b81b0a145b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e33db87b7da424f9cf494b81b0a145b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34303 } } }
I20260929 15:51:24.366953 18647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:24.366494 18646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e33db87b7da424f9cf494b81b0a145b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e33db87b7da424f9cf494b81b0a145b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34303 } } }
I20260929 15:51:24.367487 18646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:24.376380 18652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:24.380894 18652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:24.390215 18652 catalog_manager.cc:1347] Generated new cluster ID: cd7e5a966dd947988deee523a3c64e6b
I20260929 15:51:24.390362 18652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:24.414709 18652 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:51:24.416886 18652 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:24.430877 18652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e33db87b7da424f9cf494b81b0a145b: Generated new TSK 0
I20260929 15:51:24.431972 18652 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:51:24.487167 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34303
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:43437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:24.823668 18666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:24.823966 18666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:24.824064 18666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:24.831194 18666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:24.831354 18666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:24.831538 18666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:51:24.841394 18666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:43437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:24.842631 18666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:24.844017 18666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:24.858489 18671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:24.858523 18672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:24.858700 18674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:24.859500 18666 server_base.cc:1034] running on GCE node
I20260929 15:51:24.860122 18666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:24.861058 18666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:24.862313 18666 hybrid_clock.cc:648] HybridClock initialized: now 1790697084862262 us; error 43 us; skew 500 ppm
I20260929 15:51:24.862668 18666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:24 dist-test-slave-c51v krb5kdc[18560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697084, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:24.866019 18666 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:51:24.867597 18666 webserver.cc:460] Webserver started at http://127.15.196.193:45329/ using document root <none> and password file <none>
I20260929 15:51:24.868309 18666 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:24.868449 18666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:24.868748 18666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:24.871801 18666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "3bf53732e1ce48348edc5c0efd205bfd"
format_stamp: "Formatted at 2026-09-29 15:51:24 on dist-test-slave-c51v"
server_key: "24a14ce44613a3a9337c7ed7d839bf8a"
server_key_iv: "c378d24ccaa4d20b54fc874b6f7f23aa"
server_key_version: "encryptionkey@0"
I20260929 15:51:24.872705 18666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "3bf53732e1ce48348edc5c0efd205bfd"
format_stamp: "Formatted at 2026-09-29 15:51:24 on dist-test-slave-c51v"
server_key: "24a14ce44613a3a9337c7ed7d839bf8a"
server_key_iv: "c378d24ccaa4d20b54fc874b6f7f23aa"
server_key_version: "encryptionkey@0"
I20260929 15:51:24.878181 18666 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:51:24.881851 18681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:24.883229 18666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 15:51:24.883455 18666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "3bf53732e1ce48348edc5c0efd205bfd"
format_stamp: "Formatted at 2026-09-29 15:51:24 on dist-test-slave-c51v"
server_key: "24a14ce44613a3a9337c7ed7d839bf8a"
server_key_iv: "c378d24ccaa4d20b54fc874b6f7f23aa"
server_key_version: "encryptionkey@0"
I20260929 15:51:24.883661 18666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:24.897356 18666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:24.902783 18666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:24.903126 18666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:24.904233 18666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:24.905689 18666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:24.905772 18666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:24.905957 18666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:24.906020 18666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:24.944353 18666 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:33761
I20260929 15:51:24.944381 18793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:33761 every 8 connection(s)
I20260929 15:51:24.946170 18666 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:51:24.948827 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 18666
I20260929 15:51:24.949087 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:51:24.949812 16147 external_mini_cluster.cc:1428] Setting key 0e8b66ce6c398983195654fdf21395a0
Sep 29 15:51:24 dist-test-slave-c51v krb5kdc[18560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697084, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:24.969810 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:24.949047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:45669 (local address 127.15.196.254:34303)
0929 15:51:24.949574 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:24.949581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:24.950776 (+ 1195us) server_negotiation.cc:407] Connection header received
0929 15:51:24.953329 (+ 2553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:24.953333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:24.953410 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:24.953504 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:24.955850 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:24.956619 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:24.958659 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:24.959078 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:24.959231 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:24.963362 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:24.963368 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:24.963372 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:24.963407 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:24.966199 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:24.967218 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:24.967222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:24.967224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:24.967413 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:24.968024 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:24.968027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:24.968029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:24.968317 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:24.968443 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:24.969398 (+ 955us) server_negotiation.cc:299] Negotiation successful
0929 15:51:24.969564 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":246,"threads_started":1}
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
I20260929 15:51:24.971256 18797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:24.949710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:34303 (local address 127.15.196.193:45669)
0929 15:51:24.950584 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:24.950625 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:24.952984 (+ 2359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:24.953811 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:24.953828 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:24.954702 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:24.955646 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:24.955678 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:24.956782 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:24.956790 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:24.958498 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:24.958511 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:24.959376 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:24.959383 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:24.959751 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:24.960506 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:24.960530 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:24.963069 (+ 2539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:24.966362 (+ 3293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:24.966384 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:24.966405 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:24.967085 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:24.967599 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:24.967606 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:24.967612 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:24.967891 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:24.968458 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:24.968466 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:24.968964 (+ 498us) client_negotiation.cc:769] Sending connection context
0929 15:51:24.969290 (+ 326us) client_negotiation.cc:240] Negotiation successful
0929 15:51:24.969904 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":352,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
I20260929 15:51:24.973171 18794 heartbeater.cc:347] Connected to a master server at 127.15.196.254:34303
I20260929 15:51:24.973546 18794 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:24.974274 18794 heartbeater.cc:511] Master 127.15.196.254:34303 requested a full tablet report, sending...
I20260929 15:51:24.976457 18605 ts_manager.cc:194] Registered new tserver with Master: 3bf53732e1ce48348edc5c0efd205bfd (127.15.196.193:33761)
I20260929 15:51:24.978456 18605 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:45669
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:51:24.997049 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34303
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:43437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:25.317337 18800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:25.317637 18800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:25.317730 18800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:25.325019 18800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:25.325137 18800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:25.325330 18800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:51:25.335383 18800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:43437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:25.336900 18800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:25.338158 18800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:25.351684 18806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:25.351702 18805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:25.351630 18808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:25.352231 18800 server_base.cc:1034] running on GCE node
I20260929 15:51:25.353118 18800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:25.354175 18800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:25.355389 18800 hybrid_clock.cc:648] HybridClock initialized: now 1790697085355312 us; error 76 us; skew 500 ppm
I20260929 15:51:25.355700 18800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:25 dist-test-slave-c51v krb5kdc[18560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697085, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:25.358750 18800 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:51:25.360224 18800 webserver.cc:460] Webserver started at http://127.15.196.194:41609/ using document root <none> and password file <none>
I20260929 15:51:25.360934 18800 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:25.361083 18800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:25.361394 18800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:25.364559 18800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "71bc546ff939458391b4ca41fee68b15"
format_stamp: "Formatted at 2026-09-29 15:51:25 on dist-test-slave-c51v"
server_key: "83a42609f97bed5cc99caaa39182119b"
server_key_iv: "256a61b0315b1601c2b3405f0025e242"
server_key_version: "encryptionkey@0"
I20260929 15:51:25.365355 18800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "71bc546ff939458391b4ca41fee68b15"
format_stamp: "Formatted at 2026-09-29 15:51:25 on dist-test-slave-c51v"
server_key: "83a42609f97bed5cc99caaa39182119b"
server_key_iv: "256a61b0315b1601c2b3405f0025e242"
server_key_version: "encryptionkey@0"
I20260929 15:51:25.370718 18800 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 15:51:25.374478 18815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:25.375823 18800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:25.376020 18800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "71bc546ff939458391b4ca41fee68b15"
format_stamp: "Formatted at 2026-09-29 15:51:25 on dist-test-slave-c51v"
server_key: "83a42609f97bed5cc99caaa39182119b"
server_key_iv: "256a61b0315b1601c2b3405f0025e242"
server_key_version: "encryptionkey@0"
I20260929 15:51:25.376228 18800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:25.389123 18800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:25.393181 18800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:25.393492 18800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:25.394435 18800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:25.395875 18800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:25.395947 18800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:25.396029 18800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:25.396075 18800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:25.428529 18800 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:42241
I20260929 15:51:25.428565 18927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:42241 every 8 connection(s)
I20260929 15:51:25.429884 18800 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:51:25.435683 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 18800
I20260929 15:51:25.435896 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:51:25.436461 16147 external_mini_cluster.cc:1428] Setting key a98e0c23d351c776e3b68089bba83bb1
Sep 29 15:51:25 dist-test-slave-c51v krb5kdc[18560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697085, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
I20260929 15:51:25.459671 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:25.432531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:54181 (local address 127.15.196.254:34303)
0929 15:51:25.432705 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:25.432710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:25.433921 (+ 1211us) server_negotiation.cc:407] Connection header received
0929 15:51:25.435987 (+ 2066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:25.435993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:25.436080 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:25.436169 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:25.438167 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:25.439051 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:25.441754 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:25.442369 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:25.442503 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:25.447030 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:25.447040 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:25.447049 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:25.447096 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:25.456330 (+ 9234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:25.457258 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:25.457264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:25.457266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:25.457408 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:25.457859 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:25.457863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:25.457865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:25.458132 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:25.458258 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:25.459058 (+ 800us) server_negotiation.cc:299] Negotiation successful
0929 15:51:25.459359 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 15:51:25.461074 18930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:25.433042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:34303 (local address 127.15.196.194:54181)
0929 15:51:25.433737 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:25.433776 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:25.435635 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:25.436461 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:25.436475 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:25.437192 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:25.437977 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:25.437996 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:25.439218 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:25.439223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:25.441559 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:25.441578 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:25.442629 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:25.442638 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:25.442990 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:25.443864 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:25.443900 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:25.446763 (+ 2863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:25.456526 (+ 9763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:25.456535 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:25.456552 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:25.457096 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:25.457521 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:25.457524 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:25.457527 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:25.457737 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:25.458251 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:25.458258 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:25.458672 (+ 414us) client_negotiation.cc:769] Sending connection context
0929 15:51:25.458902 (+ 230us) client_negotiation.cc:240] Negotiation successful
0929 15:51:25.459502 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":270,"threads_started":1}
I20260929 15:51:25.462590 18928 heartbeater.cc:347] Connected to a master server at 127.15.196.254:34303
I20260929 15:51:25.462952 18928 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:25.463905 18928 heartbeater.cc:511] Master 127.15.196.254:34303 requested a full tablet report, sending...
I20260929 15:51:25.465526 18605 ts_manager.cc:194] Registered new tserver with Master: 71bc546ff939458391b4ca41fee68b15 (127.15.196.194:42241)
I20260929 15:51:25.466368 18605 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:54181
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:51:25.484549 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34303
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:43437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:25.806818 18933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:25.807106 18933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:25.807309 18933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:25.814738 18933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:25.814884 18933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:25.815083 18933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:51:25.824966 18933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:43437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:25.826300 18933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:25.827708 18933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:25.840821 18941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:25.840865 18939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:25.841037 18938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:25.841711 18933 server_base.cc:1034] running on GCE node
I20260929 15:51:25.842308 18933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:25.843454 18933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:25.844682 18933 hybrid_clock.cc:648] HybridClock initialized: now 1790697085844619 us; error 58 us; skew 500 ppm
I20260929 15:51:25.845026 18933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:25 dist-test-slave-c51v krb5kdc[18560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697085, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:25.848084 18933 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:51:25.849565 18933 webserver.cc:460] Webserver started at http://127.15.196.195:36581/ using document root <none> and password file <none>
I20260929 15:51:25.850265 18933 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:25.850411 18933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:25.850728 18933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:25.853667 18933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "a3d784d0596c402598596518147b133e"
format_stamp: "Formatted at 2026-09-29 15:51:25 on dist-test-slave-c51v"
server_key: "c345ac95bc94df458707f130457d3803"
server_key_iv: "001dee8d1ac1936c67dabd6bc9f2002e"
server_key_version: "encryptionkey@0"
I20260929 15:51:25.854453 18933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "a3d784d0596c402598596518147b133e"
format_stamp: "Formatted at 2026-09-29 15:51:25 on dist-test-slave-c51v"
server_key: "c345ac95bc94df458707f130457d3803"
server_key_iv: "001dee8d1ac1936c67dabd6bc9f2002e"
server_key_version: "encryptionkey@0"
I20260929 15:51:25.859539 18933 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260929 15:51:25.862897 18948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:25.864257 18933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:25.864436 18933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "a3d784d0596c402598596518147b133e"
format_stamp: "Formatted at 2026-09-29 15:51:25 on dist-test-slave-c51v"
server_key: "c345ac95bc94df458707f130457d3803"
server_key_iv: "001dee8d1ac1936c67dabd6bc9f2002e"
server_key_version: "encryptionkey@0"
I20260929 15:51:25.864637 18933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:25.878319 18933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:25.882431 18933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:25.882719 18933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:25.883728 18933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:25.885030 18933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:25.885135 18933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:25.885284 18933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:25.885356 18933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:25.918576 18933 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:33721
I20260929 15:51:25.918622 19060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:33721 every 8 connection(s)
I20260929 15:51:25.920166 18933 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:51:25.921875 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 18933
I20260929 15:51:25.922076 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:51:25.922528 16147 external_mini_cluster.cc:1428] Setting key e96f86bf96bef56fad2ddb1a6f571229
Sep 29 15:51:25 dist-test-slave-c51v krb5kdc[18560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697085, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:25.940497 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:25.923003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:55523 (local address 127.15.196.254:34303)
0929 15:51:25.923199 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:25.923203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:25.924537 (+ 1334us) server_negotiation.cc:407] Connection header received
0929 15:51:25.926374 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:25.926378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:25.926442 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:25.926520 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:25.928088 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:25.928940 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:25.930466 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:25.930880 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:25.930977 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:25.934505 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:25.934510 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:25.934513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:25.934544 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:25.937542 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:25.938357 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:25.938361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:25.938363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:25.938498 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:25.938871 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:25.938874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:25.938876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:25.939144 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:25.939325 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:25.940020 (+ 695us) server_negotiation.cc:299] Negotiation successful
0929 15:51:25.940174 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260929 15:51:25.941506 19063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:25.923554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:34303 (local address 127.15.196.195:55523)
0929 15:51:25.924369 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:25.924426 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:25.926114 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:25.926732 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:25.926740 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:25.927262 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:25.927929 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:25.927947 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:25.929078 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:25.929083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:25.930335 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:25.930347 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:25.931072 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:25.931076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:25.931271 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:25.932076 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:25.932099 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:25.934216 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:25.937674 (+ 3458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:25.937683 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:25.937701 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:25.938244 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:25.938593 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:25.938596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:25.938598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:25.938777 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:25.939335 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:25.939349 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:25.939671 (+ 322us) client_negotiation.cc:769] Sending connection context
0929 15:51:25.939913 (+ 242us) client_negotiation.cc:240] Negotiation successful
0929 15:51:25.940300 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":341,"threads_started":1}
I20260929 15:51:25.942929 19061 heartbeater.cc:347] Connected to a master server at 127.15.196.254:34303
I20260929 15:51:25.943252 19061 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:25.944001 19061 heartbeater.cc:511] Master 127.15.196.254:34303 requested a full tablet report, sending...
I20260929 15:51:25.945442 18605 ts_manager.cc:194] Registered new tserver with Master: a3d784d0596c402598596518147b133e (127.15.196.195:33721)
I20260929 15:51:25.946427 18605 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:55523
I20260929 15:51:25.950168 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 15:51:25.980934 18796 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:51:25.978203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46636 (local address 127.15.196.254:34303)
0929 15:51:25.978393 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:25.978398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:25.978863 (+ 465us) server_negotiation.cc:407] Connection header received
0929 15:51:25.979021 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:25.979025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:25.979088 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:25.979180 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:25.980562 (+ 1382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46636: BlockingRecv error: recv got EOF from 127.0.0.1:46636 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 15:51:25.982197 18794 heartbeater.cc:503] Master 127.15.196.254:34303 was elected leader, sending a full tablet report...
I20260929 15:51:25.985235 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 18666
W20260929 15:51:25.987965 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 18666: Not authorized: Client connection negotiation failed: client connection to 127.15.196.193:33761: server requires authentication, but client does not have Kerberos credentials available
I20260929 15:51:25.988173 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 18666
I20260929 15:51:26.003811 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 18800
W20260929 15:51:26.006497 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 18800: Not authorized: Client connection negotiation failed: client connection to 127.15.196.194:42241: server requires authentication, but client does not have Kerberos credentials available
I20260929 15:51:26.006680 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 18800
I20260929 15:51:26.021906 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 18933
W20260929 15:51:26.025127 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 18933: Not authorized: Client connection negotiation failed: client connection to 127.15.196.195:33721: server requires authentication, but client does not have Kerberos credentials available
W20260929 15:51:26.025292 19074 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:51:26.022938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59464 (local address 127.15.196.195:33721)
0929 15:51:26.023640 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:26.023648 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:26.023676 (+ 28us) server_negotiation.cc:407] Connection header received
0929 15:51:26.023737 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:26.023743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:26.023963 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:26.024142 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:26.024987 (+ 845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59464: BlockingRecv error: recv got EOF from 127.0.0.1:59464 (error 108)
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":408,"threads_started":1}
I20260929 15:51:26.025300 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 18933
I20260929 15:51:26.040666 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 18573
I20260929 15:51:26.041911 18635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:26.146839 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 18573
2026-09-29T15:51:26Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (5376 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:51:26 dist-test-slave-c51v krb5kdc[19078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:51:26 dist-test-slave-c51v krb5kdc[19078](info): set up 2 sockets
Sep 29 15:51:26 dist-test-slave-c51v krb5kdc[19078](info): commencing operation
krb5kdc: starting...
W20260929 15:51:27.233673 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:51:27 dist-test-slave-c51v krb5kdc[19078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:51:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:51:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:51:27.428540 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:36307
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:38951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:36307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:27.751314 19091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:27.751606 19091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:27.751695 19091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:27.758924 19091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:27.759040 19091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:27.759086 19091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:27.759145 19091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:27.759187 19091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:51:27.769156 19091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:38951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:36307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:36307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:27.770448 19091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:27.771760 19091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:27.784888 19096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:27.784889 19099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:27.784785 19097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:27.793211 19091 server_base.cc:1034] running on GCE node
I20260929 15:51:27.794080 19091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:27.795725 19091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:27.796962 19091 hybrid_clock.cc:648] HybridClock initialized: now 1790697087796921 us; error 49 us; skew 500 ppm
I20260929 15:51:27.797322 19091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:27 dist-test-slave-c51v krb5kdc[19078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697087, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:27.800591 19091 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:51:27.802184 19091 webserver.cc:460] Webserver started at http://127.15.196.254:40625/ using document root <none> and password file <none>
I20260929 15:51:27.802922 19091 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:27.803066 19091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:27.803462 19091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:27.806634 19091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "490f25b4260046a5b56c55d1e171978f"
format_stamp: "Formatted at 2026-09-29 15:51:27 on dist-test-slave-c51v"
server_key: "9a6e3789296deb9e68950a483a3e693c"
server_key_iv: "09cd95e10a83e461a4dfe6b1f060b3b8"
server_key_version: "encryptionkey@0"
I20260929 15:51:27.807590 19091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "490f25b4260046a5b56c55d1e171978f"
format_stamp: "Formatted at 2026-09-29 15:51:27 on dist-test-slave-c51v"
server_key: "9a6e3789296deb9e68950a483a3e693c"
server_key_iv: "09cd95e10a83e461a4dfe6b1f060b3b8"
server_key_version: "encryptionkey@0"
I20260929 15:51:27.812844 19091 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260929 15:51:27.816439 19106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:27.817833 19091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:51:27.818022 19091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "490f25b4260046a5b56c55d1e171978f"
format_stamp: "Formatted at 2026-09-29 15:51:27 on dist-test-slave-c51v"
server_key: "9a6e3789296deb9e68950a483a3e693c"
server_key_iv: "09cd95e10a83e461a4dfe6b1f060b3b8"
server_key_version: "encryptionkey@0"
I20260929 15:51:27.818230 19091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:27.844162 19091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:27.848830 19091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:27.849143 19091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:27.867136 19091 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:36307
I20260929 15:51:27.867153 19157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:36307 every 8 connection(s)
I20260929 15:51:27.868582 19091 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:51:27.872620 19158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:27.877012 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 19091
I20260929 15:51:27.877233 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:51:27.877884 16147 external_mini_cluster.cc:1428] Setting key b0441da30347c1b442bf206210144316
I20260929 15:51:27.885095 19158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f: Bootstrap starting.
I20260929 15:51:27.888863 19158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:27.890185 19158 log.cc:824] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:27.893105 19158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f: No bootstrap required, opened a new log
Sep 29 15:51:27 dist-test-slave-c51v krb5kdc[19078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:27.903322 19161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:27.879890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52384 (local address 127.15.196.254:36307)
0929 15:51:27.880816 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:27.880832 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:27.880884 (+ 52us) server_negotiation.cc:407] Connection header received
0929 15:51:27.883465 (+ 2581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:27.883498 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:27.884039 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:27.884504 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:27.885951 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:27.888030 (+ 2079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:27.889320 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:27.890412 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:27.890594 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:27.894541 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:27.894552 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:27.894572 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:27.894618 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:27.898640 (+ 4022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:27.899322 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:27.899330 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:27.899339 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:27.899566 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:27.899920 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:27.899925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:27.899929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:27.900690 (+ 761us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:27.901020 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:27.901334 (+ 314us) server_negotiation.cc:299] Negotiation successful
0929 15:51:27.901831 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":414,"threads_started":1}
I20260929 15:51:27.904119 19158 raft_consensus.cc:348] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490f25b4260046a5b56c55d1e171978f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36307 } }
I20260929 15:51:27.904554 19158 raft_consensus.cc:374] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:27.904659 19158 raft_consensus.cc:729] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 490f25b4260046a5b56c55d1e171978f, State: Initialized, Role: FOLLOWER
I20260929 15:51:27.905342 19158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490f25b4260046a5b56c55d1e171978f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36307 } }
I20260929 15:51:27.905591 19158 raft_consensus.cc:388] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:51:27.905740 19158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:51:27.905952 19158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:27.907836 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:51:27.908898 19158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490f25b4260046a5b56c55d1e171978f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36307 } }
I20260929 15:51:27.909320 19158 leader_election.cc:302] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 490f25b4260046a5b56c55d1e171978f; no voters:
I20260929 15:51:27.909929 19158 leader_election.cc:288] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:51:27.910033 19163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:27.912554 19158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:27.917004 19163 raft_consensus.cc:686] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [term 1 LEADER]: Becoming Leader. State: Replica: 490f25b4260046a5b56c55d1e171978f, State: Running, Role: LEADER
I20260929 15:51:27.917801 19163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490f25b4260046a5b56c55d1e171978f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36307 } }
I20260929 15:51:27.921321 19164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "490f25b4260046a5b56c55d1e171978f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490f25b4260046a5b56c55d1e171978f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36307 } } }
I20260929 15:51:27.921793 19164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:27.922429 19165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 490f25b4260046a5b56c55d1e171978f. Latest consensus state: current_term: 1 leader_uuid: "490f25b4260046a5b56c55d1e171978f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490f25b4260046a5b56c55d1e171978f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36307 } } }
I20260929 15:51:27.922740 19165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:27.924391 19174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:27.934131 19174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:27.943387 19183 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 15:51:27.946398 19174 catalog_manager.cc:1347] Generated new cluster ID: da942b8bf05c44f2953f55899f70c2a4
I20260929 15:51:27.946537 19174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:27.962500 19161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:27.952905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52386 (local address 127.15.196.254:36307)
0929 15:51:27.953083 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:27.953091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:27.953751 (+ 660us) server_negotiation.cc:407] Connection header received
0929 15:51:27.953927 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:27.953933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:27.954039 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:27.954163 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:27.955345 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:27.956661 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:27.958405 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:27.959121 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:27.959265 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:27.960469 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:27.960477 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:27.960483 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:27.961407 (+ 924us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:27.961485 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:27.961523 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:27.961650 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:27.961784 (+ 134us) server_negotiation.cc:299] Negotiation successful
0929 15:51:27.961883 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 15:51:27.984737 19174 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:51:27.986361 19174 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:27.993232 19174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 490f25b4260046a5b56c55d1e171978f: Generated new TSK 0
I20260929 15:51:27.994046 19174 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:51:28.002408 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 19091
I20260929 15:51:28.003652 19153 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:28.096226 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 19091
2026-09-29T15:51:28Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1944 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:51:28 dist-test-slave-c51v krb5kdc[19193](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:51:28 dist-test-slave-c51v krb5kdc[19193](info): set up 2 sockets
Sep 29 15:51:28 dist-test-slave-c51v krb5kdc[19193](info): commencing operation
krb5kdc: starting...
W20260929 15:51:29.213667 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.002s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:51:29 dist-test-slave-c51v krb5kdc[19193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:51:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:51:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:51:29.408913 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:46227
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:46227
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:29.732543 19206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:29.732969 19206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:29.733134 19206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:29.740598 19206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:29.740748 19206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:29.740895 19206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:29.740994 19206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:29.741080 19206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:51:29.751310 19206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:46227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:46227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:29.752807 19206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:29.754217 19206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:29.766584 19211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:29.766646 19212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:29.766680 19214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:29.767899 19206 server_base.cc:1034] running on GCE node
I20260929 15:51:29.768718 19206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:29.770159 19206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:29.771414 19206 hybrid_clock.cc:648] HybridClock initialized: now 1790697089771371 us; error 49 us; skew 500 ppm
I20260929 15:51:29.771772 19206 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:29 dist-test-slave-c51v krb5kdc[19193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697089, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:29.774904 19206 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:51:29.776535 19206 webserver.cc:460] Webserver started at http://127.15.196.254:43799/ using document root <none> and password file <none>
I20260929 15:51:29.777237 19206 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:29.777386 19206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:29.777709 19206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:29.780874 19206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "ca768eb7611045feba36c7e14827dac3"
format_stamp: "Formatted at 2026-09-29 15:51:29 on dist-test-slave-c51v"
server_key: "9f56aeb2dbef293df24207fc7e6c20f5"
server_key_iv: "aac6f5fc86a3ef335152c5a8835d6665"
server_key_version: "encryptionkey@0"
I20260929 15:51:29.781679 19206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "ca768eb7611045feba36c7e14827dac3"
format_stamp: "Formatted at 2026-09-29 15:51:29 on dist-test-slave-c51v"
server_key: "9f56aeb2dbef293df24207fc7e6c20f5"
server_key_iv: "aac6f5fc86a3ef335152c5a8835d6665"
server_key_version: "encryptionkey@0"
I20260929 15:51:29.787074 19206 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 15:51:29.790514 19221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:29.792076 19206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:29.792268 19206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "ca768eb7611045feba36c7e14827dac3"
format_stamp: "Formatted at 2026-09-29 15:51:29 on dist-test-slave-c51v"
server_key: "9f56aeb2dbef293df24207fc7e6c20f5"
server_key_iv: "aac6f5fc86a3ef335152c5a8835d6665"
server_key_version: "encryptionkey@0"
I20260929 15:51:29.792482 19206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:29.812980 19206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:29.817407 19206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:29.817723 19206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:29.836763 19206 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:46227
I20260929 15:51:29.836869 19272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:46227 every 8 connection(s)
I20260929 15:51:29.838232 19206 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:51:29.842072 19273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:29.847368 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 19206
I20260929 15:51:29.847589 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:51:29.848351 16147 external_mini_cluster.cc:1428] Setting key b57c8498f1c50317d8682dd654460adf
I20260929 15:51:29.853790 19273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3: Bootstrap starting.
I20260929 15:51:29.856967 19273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:29.858032 19273 log.cc:824] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:29.861842 19273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3: No bootstrap required, opened a new log
Sep 29 15:51:29 dist-test-slave-c51v krb5kdc[19193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:29.872786 19273 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca768eb7611045feba36c7e14827dac3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46227 } }
I20260929 15:51:29.873281 19273 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:29.873358 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:29.850644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46054 (local address 127.15.196.254:46227)
0929 15:51:29.851535 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:29.851548 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:29.851596 (+ 48us) server_negotiation.cc:407] Connection header received
0929 15:51:29.853279 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:29.853302 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:29.853711 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:29.854145 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:29.855630 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:29.856934 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:29.858188 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:29.859359 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:29.859563 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:29.864304 (+ 4741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:29.864315 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:29.864336 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:29.864408 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:29.868594 (+ 4186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:29.869306 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:29.869315 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:29.869325 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:29.869554 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:29.869919 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:29.869923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:29.869926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:29.870739 (+ 813us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:29.871051 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:29.871473 (+ 422us) server_negotiation.cc:299] Negotiation successful
0929 15:51:29.871964 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":341,"threads_started":1}
I20260929 15:51:29.873425 19273 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca768eb7611045feba36c7e14827dac3, State: Initialized, Role: FOLLOWER
I20260929 15:51:29.874047 19273 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca768eb7611045feba36c7e14827dac3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46227 } }
I20260929 15:51:29.874308 19273 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:51:29.874457 19273 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:51:29.874663 19273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:29.877125 19273 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca768eb7611045feba36c7e14827dac3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46227 } }
I20260929 15:51:29.877672 19273 leader_election.cc:302] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca768eb7611045feba36c7e14827dac3; no voters:
I20260929 15:51:29.878242 19273 leader_election.cc:288] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:51:29.878387 19278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:29.879905 19278 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [term 1 LEADER]: Becoming Leader. State: Replica: ca768eb7611045feba36c7e14827dac3, State: Running, Role: LEADER
I20260929 15:51:29.880487 19278 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca768eb7611045feba36c7e14827dac3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46227 } }
I20260929 15:51:29.880584 19273 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:29.883728 19280 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca768eb7611045feba36c7e14827dac3. Latest consensus state: current_term: 1 leader_uuid: "ca768eb7611045feba36c7e14827dac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca768eb7611045feba36c7e14827dac3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46227 } } }
I20260929 15:51:29.884182 19280 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:29.884336 19279 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca768eb7611045feba36c7e14827dac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca768eb7611045feba36c7e14827dac3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46227 } } }
I20260929 15:51:29.884647 19279 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3 [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:29.887208 19287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:29.892694 19287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:29.901795 19287 catalog_manager.cc:1347] Generated new cluster ID: 0dcf64aa54434451a526f8ef3217d88b
I20260929 15:51:29.901924 19287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:29.915756 19287 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:51:29.917168 19287 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:29.924870 19287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca768eb7611045feba36c7e14827dac3: Generated new TSK 0
I20260929 15:51:29.925773 19287 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:51:29.978809 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46227
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:30.308943 19299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:30.309298 19299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:30.309464 19299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:30.316766 19299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:30.316917 19299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:30.317086 19299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:51:30.327175 19299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:30.328589 19299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:30.329957 19299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:30.344038 19305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:30.344090 19304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:30.344123 19307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:30.345024 19299 server_base.cc:1034] running on GCE node
I20260929 15:51:30.345715 19299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:30.346710 19299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:30.347968 19299 hybrid_clock.cc:648] HybridClock initialized: now 1790697090347892 us; error 60 us; skew 500 ppm
I20260929 15:51:30.348306 19299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:30 dist-test-slave-c51v krb5kdc[19193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697090, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:30.351613 19299 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:51:30.353080 19299 webserver.cc:460] Webserver started at http://127.15.196.193:41053/ using document root <none> and password file <none>
I20260929 15:51:30.353783 19299 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:30.353935 19299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:30.354242 19299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:30.357290 19299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "a3800fb7beb6419abf79e0c503f37f1d"
format_stamp: "Formatted at 2026-09-29 15:51:30 on dist-test-slave-c51v"
server_key: "a0e8deffa59197fd65660eb4da392816"
server_key_iv: "a6457250f60a91f9f0d391e55e83433b"
server_key_version: "encryptionkey@0"
I20260929 15:51:30.358095 19299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "a3800fb7beb6419abf79e0c503f37f1d"
format_stamp: "Formatted at 2026-09-29 15:51:30 on dist-test-slave-c51v"
server_key: "a0e8deffa59197fd65660eb4da392816"
server_key_iv: "a6457250f60a91f9f0d391e55e83433b"
server_key_version: "encryptionkey@0"
I20260929 15:51:30.363511 19299 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 15:51:30.366986 19314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:30.368371 19299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:30.368501 19299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "a3800fb7beb6419abf79e0c503f37f1d"
format_stamp: "Formatted at 2026-09-29 15:51:30 on dist-test-slave-c51v"
server_key: "a0e8deffa59197fd65660eb4da392816"
server_key_iv: "a6457250f60a91f9f0d391e55e83433b"
server_key_version: "encryptionkey@0"
I20260929 15:51:30.368662 19299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:30.388659 19299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:30.392570 19299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:30.392812 19299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:30.393766 19299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:30.395058 19299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:30.395133 19299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:30.395216 19299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:30.395259 19299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:30.429086 19299 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:38937
I20260929 15:51:30.429174 19426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:38937 every 8 connection(s)
I20260929 15:51:30.430424 19299 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:51:30.438645 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 19299
I20260929 15:51:30.438858 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:51:30.439531 16147 external_mini_cluster.cc:1428] Setting key 8ac2f4d58fbbbdd74f4c249ef013023c
Sep 29 15:51:30 dist-test-slave-c51v krb5kdc[19193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697090, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:30.450703 19429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:30.433007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:60783 (local address 127.15.196.254:46227)
0929 15:51:30.433463 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:30.433468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:30.434311 (+ 843us) server_negotiation.cc:407] Connection header received
0929 15:51:30.436199 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:30.436205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:30.436311 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:30.436442 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:30.437975 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:30.438718 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:30.440658 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:30.441086 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:30.441227 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:30.444894 (+ 3667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:30.444906 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:30.444909 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:30.444944 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:30.447623 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:30.448400 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:30.448406 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:30.448408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:30.448522 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:30.448961 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:30.448966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:30.448969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:30.449230 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:30.449389 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:30.450115 (+ 726us) server_negotiation.cc:299] Negotiation successful
0929 15:51:30.450337 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":246,"threads_started":1}
I20260929 15:51:30.451807 19430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:30.433449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:46227 (local address 127.15.196.193:60783)
0929 15:51:30.434151 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:30.434196 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:30.435928 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:30.436685 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:30.436694 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:30.437219 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:30.437799 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:30.437815 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:30.438955 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:30.438959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:30.440507 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:30.440520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:30.441318 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:30.441328 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:30.441568 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:30.442250 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:30.442277 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:30.444655 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:30.447781 (+ 3126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:30.447790 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:30.447808 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:30.448261 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:30.448632 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:30.448637 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:30.448641 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:30.448837 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:30.449359 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:30.449368 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:30.449733 (+ 365us) client_negotiation.cc:769] Sending connection context
0929 15:51:30.450007 (+ 274us) client_negotiation.cc:240] Negotiation successful
0929 15:51:30.450581 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":257,"threads_started":1}
I20260929 15:51:30.453567 19427 heartbeater.cc:347] Connected to a master server at 127.15.196.254:46227
I20260929 15:51:30.453871 19427 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:30.454609 19427 heartbeater.cc:511] Master 127.15.196.254:46227 requested a full tablet report, sending...
I20260929 15:51:30.457072 19238 ts_manager.cc:194] Registered new tserver with Master: a3800fb7beb6419abf79e0c503f37f1d (127.15.196.193:38937)
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
I20260929 15:51:30.458721 19238 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:60783
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:51:30.483268 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46227
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:30.806871 19433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:30.807199 19433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:30.807402 19433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:30.814496 19433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:30.814651 19433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:30.814841 19433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:51:30.824482 19433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:30.825860 19433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:30.827204 19433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:30.840150 19438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:30.840217 19441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:30.840755 19439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:30.842187 19433 server_base.cc:1034] running on GCE node
I20260929 15:51:30.842753 19433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:30.843770 19433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:30.844990 19433 hybrid_clock.cc:648] HybridClock initialized: now 1790697090844936 us; error 47 us; skew 500 ppm
I20260929 15:51:30.845299 19433 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:30 dist-test-slave-c51v krb5kdc[19193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697090, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:30.853235 19433 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:51:30.854808 19433 webserver.cc:460] Webserver started at http://127.15.196.194:35737/ using document root <none> and password file <none>
I20260929 15:51:30.855628 19433 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:30.855773 19433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:30.856078 19433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:30.859031 19433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "11a1a33dbdf24f3da70d57d34224a103"
format_stamp: "Formatted at 2026-09-29 15:51:30 on dist-test-slave-c51v"
server_key: "b4d2ba2b9e75c4a71afacc951834223d"
server_key_iv: "0ce08a8a9edd9cba2360b2542a936305"
server_key_version: "encryptionkey@0"
I20260929 15:51:30.859951 19433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "11a1a33dbdf24f3da70d57d34224a103"
format_stamp: "Formatted at 2026-09-29 15:51:30 on dist-test-slave-c51v"
server_key: "b4d2ba2b9e75c4a71afacc951834223d"
server_key_iv: "0ce08a8a9edd9cba2360b2542a936305"
server_key_version: "encryptionkey@0"
I20260929 15:51:30.865113 19433 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:51:30.868667 19448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:30.870013 19433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260929 15:51:30.870184 19433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "11a1a33dbdf24f3da70d57d34224a103"
format_stamp: "Formatted at 2026-09-29 15:51:30 on dist-test-slave-c51v"
server_key: "b4d2ba2b9e75c4a71afacc951834223d"
server_key_iv: "0ce08a8a9edd9cba2360b2542a936305"
server_key_version: "encryptionkey@0"
I20260929 15:51:30.870393 19433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:30.891278 19433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:30.895911 19433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:30.896209 19433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:30.897249 19433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:30.898597 19433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:30.898707 19433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:30.898861 19433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:30.898936 19433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:30.931928 19433 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:32949
I20260929 15:51:30.931994 19560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:32949 every 8 connection(s)
I20260929 15:51:30.933310 19433 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:51:30.941773 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 19433
I20260929 15:51:30.941972 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:51:30.942622 16147 external_mini_cluster.cc:1428] Setting key 9ef89001b45fee8d30d0e6bf321e0817
Sep 29 15:51:30 dist-test-slave-c51v krb5kdc[19193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697090, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:30.956593 19429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:30.935938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:45275 (local address 127.15.196.254:46227)
0929 15:51:30.936149 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:30.936155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:30.937380 (+ 1225us) server_negotiation.cc:407] Connection header received
0929 15:51:30.939251 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:30.939257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:30.939369 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:30.939541 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:30.942127 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:30.943393 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:30.945947 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:30.946595 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:30.946736 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:30.950127 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:30.950136 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:30.950144 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:30.950190 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:30.953168 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:30.954118 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:30.954122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:30.954124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:30.954265 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:30.954802 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:30.954805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:30.954807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:30.955081 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:30.955196 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:30.956182 (+ 986us) server_negotiation.cc:299] Negotiation successful
0929 15:51:30.956355 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 15:51:30.958250 19563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:30.936460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:46227 (local address 127.15.196.194:45275)
0929 15:51:30.937210 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:30.937256 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:30.938931 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:30.939924 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:30.939952 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:30.940882 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:30.941799 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:30.941827 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:30.943611 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:30.943616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:30.945793 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:30.945807 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:30.946809 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:30.946814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:30.947018 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:30.947768 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:30.947796 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:30.949894 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:30.953309 (+ 3415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:30.953321 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:30.953343 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:30.953983 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:30.954388 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:30.954396 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:30.954409 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:30.954658 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:30.955223 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:30.955234 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:30.955752 (+ 518us) client_negotiation.cc:769] Sending connection context
0929 15:51:30.956093 (+ 341us) client_negotiation.cc:240] Negotiation successful
0929 15:51:30.956697 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":313,"threads_started":1}
I20260929 15:51:30.959750 19561 heartbeater.cc:347] Connected to a master server at 127.15.196.254:46227
I20260929 15:51:30.960101 19561 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:30.961091 19561 heartbeater.cc:511] Master 127.15.196.254:46227 requested a full tablet report, sending...
I20260929 15:51:30.962924 19238 ts_manager.cc:194] Registered new tserver with Master: 11a1a33dbdf24f3da70d57d34224a103 (127.15.196.194:32949)
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
I20260929 15:51:30.963871 19238 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:45275
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:51:30.988705 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46227
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:31.314507 19566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:31.314772 19566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:31.314865 19566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:31.322178 19566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:31.322288 19566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:31.322408 19566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:51:31.332242 19566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:31.333478 19566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:31.334722 19566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:31.348097 19571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:31.348203 19572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:31.348165 19574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:31.348908 19566 server_base.cc:1034] running on GCE node
I20260929 15:51:31.349562 19566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:31.350474 19566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:31.351699 19566 hybrid_clock.cc:648] HybridClock initialized: now 1790697091351651 us; error 70 us; skew 500 ppm
I20260929 15:51:31.352025 19566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:31 dist-test-slave-c51v krb5kdc[19193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697091, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:31.355211 19566 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:51:31.356741 19566 webserver.cc:460] Webserver started at http://127.15.196.195:35005/ using document root <none> and password file <none>
I20260929 15:51:31.357441 19566 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:31.357597 19566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:31.357901 19566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:31.360885 19566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "ae34283d16a6472dbbe500bf4b577c48"
format_stamp: "Formatted at 2026-09-29 15:51:31 on dist-test-slave-c51v"
server_key: "a28a9c01efc86c193c02ef202e4a1b7b"
server_key_iv: "4530442e6ff51dd85ccf9c254ced0e08"
server_key_version: "encryptionkey@0"
I20260929 15:51:31.361708 19566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "ae34283d16a6472dbbe500bf4b577c48"
format_stamp: "Formatted at 2026-09-29 15:51:31 on dist-test-slave-c51v"
server_key: "a28a9c01efc86c193c02ef202e4a1b7b"
server_key_iv: "4530442e6ff51dd85ccf9c254ced0e08"
server_key_version: "encryptionkey@0"
I20260929 15:51:31.367399 19566 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260929 15:51:31.371040 19581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:31.372630 19566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:31.372797 19566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "ae34283d16a6472dbbe500bf4b577c48"
format_stamp: "Formatted at 2026-09-29 15:51:31 on dist-test-slave-c51v"
server_key: "a28a9c01efc86c193c02ef202e4a1b7b"
server_key_iv: "4530442e6ff51dd85ccf9c254ced0e08"
server_key_version: "encryptionkey@0"
I20260929 15:51:31.372980 19566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:31.403462 19566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:31.407826 19566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:31.408142 19566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:31.409147 19566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:31.410530 19566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:31.410602 19566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:31.410749 19566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:31.410825 19566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:31.446345 19566 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:41243
I20260929 15:51:31.446408 19693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:41243 every 8 connection(s)
I20260929 15:51:31.447811 19566 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:51:31.458261 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 19566
I20260929 15:51:31.458461 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:51:31.459028 16147 external_mini_cluster.cc:1428] Setting key 88a0b62bc5e246331628c50a04603151
Sep 29 15:51:31 dist-test-slave-c51v krb5kdc[19193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697091, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:31.462455 19427 heartbeater.cc:503] Master 127.15.196.254:46227 was elected leader, sending a full tablet report...
I20260929 15:51:31.467767 19429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:31.450335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:59069 (local address 127.15.196.254:46227)
0929 15:51:31.450511 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:31.450516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:31.451714 (+ 1198us) server_negotiation.cc:407] Connection header received
0929 15:51:31.453433 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:31.453438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:31.453531 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:31.453617 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:31.455023 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:31.455862 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:31.457334 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:31.457739 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:31.457830 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:31.461783 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:31.461794 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:31.461797 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:31.461829 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:31.464802 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:31.465504 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:31.465511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:31.465515 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:31.465642 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:31.466066 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:31.466072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:31.466076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:31.466347 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:31.466514 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:31.467183 (+ 669us) server_negotiation.cc:299] Negotiation successful
0929 15:51:31.467431 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 15:51:31.468716 19696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:31.450847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:46227 (local address 127.15.196.195:59069)
0929 15:51:31.451564 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:31.451601 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:31.453181 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:31.453819 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:31.453826 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:31.454329 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:31.454868 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:31.454896 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:31.455994 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:31.455998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:31.457209 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:31.457220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:31.458001 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:31.458011 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:31.458312 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:31.459018 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:31.459051 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:31.461584 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:31.464964 (+ 3380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:31.464972 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:31.464995 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:31.465383 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:31.465751 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:31.465756 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:31.465759 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:31.465954 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:31.466483 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:31.466490 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:31.466823 (+ 333us) client_negotiation.cc:769] Sending connection context
0929 15:51:31.467086 (+ 263us) client_negotiation.cc:240] Negotiation successful
0929 15:51:31.467623 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":258,"threads_started":1}
I20260929 15:51:31.470018 19694 heartbeater.cc:347] Connected to a master server at 127.15.196.254:46227
I20260929 15:51:31.470310 19694 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:31.471019 19694 heartbeater.cc:511] Master 127.15.196.254:46227 requested a full tablet report, sending...
I20260929 15:51:31.472504 19238 ts_manager.cc:194] Registered new tserver with Master: ae34283d16a6472dbbe500bf4b577c48 (127.15.196.195:41243)
I20260929 15:51:31.473434 19238 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:59069
I20260929 15:51:31.474546 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 15:51:31 dist-test-slave-c51v krb5kdc[19193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697091, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 29 15:51:31 dist-test-slave-c51v krb5kdc[19193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697091, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:31.523530 19429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:31.509449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.15.196.254:46227)
0929 15:51:31.509654 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:31.509663 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:31.510097 (+ 434us) server_negotiation.cc:407] Connection header received
0929 15:51:31.510310 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:31.510315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:31.510378 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:31.510491 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:31.511935 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:31.512878 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:31.514130 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:31.514522 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:31.514654 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:31.518503 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:31.518508 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:31.518512 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:31.518545 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:31.520969 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:31.521594 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:31.521605 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:31.521607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:31.521730 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:31.522109 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:31.522116 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:31.522119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:31.522441 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:31.522684 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:31.522961 (+ 277us) server_negotiation.cc:299] Negotiation successful
0929 15:51:31.523176 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260929 15:51:31.524046 19238 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46068
I20260929 15:51:31.526008 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 19299
Sep 29 15:51:31 dist-test-slave-c51v krb5kdc[19193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697091, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:51:31.541800 19705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:31.526997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35710 (local address 127.15.196.193:38937)
0929 15:51:31.527643 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:31.527654 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:31.527685 (+ 31us) server_negotiation.cc:407] Connection header received
0929 15:51:31.527738 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:31.527744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:31.527999 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:31.528167 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:31.529337 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:31.530347 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:31.531636 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:31.532204 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:31.532353 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:31.536250 (+ 3897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:31.536262 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:31.536277 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:31.536308 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:31.538954 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:31.539717 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:31.539724 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:31.539726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:31.539876 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:31.540257 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:31.540267 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:31.540270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:31.540588 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:31.540906 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:31.541165 (+ 259us) server_negotiation.cc:299] Negotiation successful
0929 15:51:31.541452 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":294,"threads_started":1}
W20260929 15:51:31.543010 19422 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35710
W20260929 15:51:31.544183 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 19299: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:51:31.544386 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 19299
I20260929 15:51:31.559742 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 19433
Sep 29 15:51:31 dist-test-slave-c51v krb5kdc[19193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697091, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:51:31.575665 19706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:31.560607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34802 (local address 127.15.196.194:32949)
0929 15:51:31.561211 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:31.561218 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:31.561244 (+ 26us) server_negotiation.cc:407] Connection header received
0929 15:51:31.561305 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:31.561312 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:31.561523 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:31.561662 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:31.562802 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:31.563866 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:31.565271 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:31.565825 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:31.565935 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:31.570136 (+ 4201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:31.570148 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:31.570162 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:31.570207 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:31.572942 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:31.573635 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:31.573642 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:31.573644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:31.573765 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:31.574082 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:31.574086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:31.574088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:31.574520 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:31.574749 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:31.574989 (+ 240us) server_negotiation.cc:299] Negotiation successful
0929 15:51:31.575273 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":330,"threads_started":1}
W20260929 15:51:31.577003 19556 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34802
W20260929 15:51:31.578178 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 19433: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:51:31.578349 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 19433
I20260929 15:51:31.594182 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 19566
Sep 29 15:51:31 dist-test-slave-c51v krb5kdc[19193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697091, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:51:31.610785 19707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:31.595390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43906 (local address 127.15.196.195:41243)
0929 15:51:31.596023 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:31.596030 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:31.596058 (+ 28us) server_negotiation.cc:407] Connection header received
0929 15:51:31.596118 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:31.596124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:31.596347 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:31.596549 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:31.597939 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:31.599113 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:31.600453 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:31.600998 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:31.601095 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:31.605338 (+ 4243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:31.605350 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:31.605365 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:31.605400 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:31.608368 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:31.608989 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:31.608996 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:31.608998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:31.609113 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:31.609420 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:31.609423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:31.609425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:31.609745 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:31.609974 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:31.610180 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 15:51:31.610372 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":331,"threads_started":1}
W20260929 15:51:31.612139 19689 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43906
W20260929 15:51:31.613221 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 19566: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:51:31.613396 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 19566
I20260929 15:51:31.628746 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 19206
I20260929 15:51:31.629961 19268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:31.751145 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 19206
2026-09-29T15:51:31Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3660 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:51:31 dist-test-slave-c51v krb5kdc[19711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:51:31 dist-test-slave-c51v krb5kdc[19711](info): set up 2 sockets
Sep 29 15:51:31 dist-test-slave-c51v krb5kdc[19711](info): commencing operation
krb5kdc: starting...
W20260929 15:51:32.841668 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:51:32 dist-test-slave-c51v krb5kdc[19711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:51:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:51:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:51:33.039451 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:34853
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:45899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:34853
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:33.365434 19724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:33.365727 19724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:33.365901 19724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:33.373167 19724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:33.373311 19724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:33.373399 19724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:33.373498 19724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:33.373585 19724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:51:33.383543 19724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:45899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:34853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:34853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:33.384882 19724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:33.386214 19724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:33.398370 19729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:33.398530 19730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:33.398542 19732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:33.400884 19724 server_base.cc:1034] running on GCE node
I20260929 15:51:33.401816 19724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:33.403460 19724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:33.404713 19724 hybrid_clock.cc:648] HybridClock initialized: now 1790697093404676 us; error 50 us; skew 500 ppm
I20260929 15:51:33.405144 19724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:33 dist-test-slave-c51v krb5kdc[19711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697093, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:33.408861 19724 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:51:33.410750 19724 webserver.cc:460] Webserver started at http://127.15.196.254:44745/ using document root <none> and password file <none>
I20260929 15:51:33.411665 19724 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:33.411828 19724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:33.412180 19724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:33.415589 19724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "d357a9d90483453f83be79a1ca59ccda"
format_stamp: "Formatted at 2026-09-29 15:51:33 on dist-test-slave-c51v"
server_key: "ed9847a6feabbb1cfbd8aa4b0ced9b5f"
server_key_iv: "3fd7b6a5e178799db3d6c9fde0a699ac"
server_key_version: "encryptionkey@0"
I20260929 15:51:33.416491 19724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "d357a9d90483453f83be79a1ca59ccda"
format_stamp: "Formatted at 2026-09-29 15:51:33 on dist-test-slave-c51v"
server_key: "ed9847a6feabbb1cfbd8aa4b0ced9b5f"
server_key_iv: "3fd7b6a5e178799db3d6c9fde0a699ac"
server_key_version: "encryptionkey@0"
I20260929 15:51:33.422528 19724 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260929 15:51:33.426538 19739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:33.428025 19724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 15:51:33.428210 19724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "d357a9d90483453f83be79a1ca59ccda"
format_stamp: "Formatted at 2026-09-29 15:51:33 on dist-test-slave-c51v"
server_key: "ed9847a6feabbb1cfbd8aa4b0ced9b5f"
server_key_iv: "3fd7b6a5e178799db3d6c9fde0a699ac"
server_key_version: "encryptionkey@0"
I20260929 15:51:33.428427 19724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:33.450682 19724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:33.455111 19724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:33.455462 19724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:33.473779 19724 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:34853
I20260929 15:51:33.473803 19790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:34853 every 8 connection(s)
I20260929 15:51:33.475173 19724 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:51:33.478955 19791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:33.478993 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 19724
I20260929 15:51:33.479234 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:51:33.479900 16147 external_mini_cluster.cc:1428] Setting key c7b26d8cd4819136d1f2806126c7b175
I20260929 15:51:33.493636 19791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda: Bootstrap starting.
Sep 29 15:51:33 dist-test-slave-c51v krb5kdc[19711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:33.497694 19791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:33.499115 19791 log.cc:824] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:33.502432 19791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda: No bootstrap required, opened a new log
I20260929 15:51:33.504911 19794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:33.481874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54390 (local address 127.15.196.254:34853)
0929 15:51:33.482742 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:33.482754 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:33.482796 (+ 42us) server_negotiation.cc:407] Connection header received
0929 15:51:33.484908 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:33.484935 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:33.485340 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:33.485670 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:33.487539 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:33.488810 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:33.490896 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:33.491678 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:33.491819 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:33.496889 (+ 5070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:33.496899 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:33.496916 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:33.496955 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:33.500448 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:33.501246 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:33.501253 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:33.501260 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:33.501429 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:33.501889 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:33.501894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:33.501897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:33.502453 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:33.502727 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:33.503362 (+ 635us) server_negotiation.cc:299] Negotiation successful
0929 15:51:33.503783 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":390,"threads_started":1}
I20260929 15:51:33.511969 19791 raft_consensus.cc:348] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d357a9d90483453f83be79a1ca59ccda" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34853 } }
I20260929 15:51:33.512410 19791 raft_consensus.cc:374] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:33.512554 19791 raft_consensus.cc:729] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d357a9d90483453f83be79a1ca59ccda, State: Initialized, Role: FOLLOWER
I20260929 15:51:33.513206 19791 consensus_queue.cc:260] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d357a9d90483453f83be79a1ca59ccda" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34853 } }
I20260929 15:51:33.513458 19791 raft_consensus.cc:388] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:51:33.513608 19791 raft_consensus.cc:482] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:51:33.513803 19791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:33.516387 19791 raft_consensus.cc:504] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d357a9d90483453f83be79a1ca59ccda" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34853 } }
I20260929 15:51:33.516999 19791 leader_election.cc:302] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d357a9d90483453f83be79a1ca59ccda; no voters:
I20260929 15:51:33.517612 19791 leader_election.cc:288] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:51:33.517752 19796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:33.519448 19796 raft_consensus.cc:686] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [term 1 LEADER]: Becoming Leader. State: Replica: d357a9d90483453f83be79a1ca59ccda, State: Running, Role: LEADER
I20260929 15:51:33.520043 19796 consensus_queue.cc:237] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d357a9d90483453f83be79a1ca59ccda" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34853 } }
I20260929 15:51:33.520221 19791 sys_catalog.cc:564] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:33.523160 19797 sys_catalog.cc:455] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d357a9d90483453f83be79a1ca59ccda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d357a9d90483453f83be79a1ca59ccda" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34853 } } }
I20260929 15:51:33.523134 19798 sys_catalog.cc:455] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [sys.catalog]: SysCatalogTable state changed. Reason: New leader d357a9d90483453f83be79a1ca59ccda. Latest consensus state: current_term: 1 leader_uuid: "d357a9d90483453f83be79a1ca59ccda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d357a9d90483453f83be79a1ca59ccda" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34853 } } }
I20260929 15:51:33.523623 19797 sys_catalog.cc:458] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:33.523623 19798 sys_catalog.cc:458] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:33.524580 19802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:33.531082 19802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:33.540318 19802 catalog_manager.cc:1347] Generated new cluster ID: a8dcad9447d74174b5c856eadea1961a
I20260929 15:51:33.540449 19802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:33.575690 19802 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:51:33.577160 19802 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:33.585891 19802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d357a9d90483453f83be79a1ca59ccda: Generated new TSK 0
I20260929 15:51:33.586764 19802 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:51:33.646555 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34853
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:45899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:34.011267 19817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:34.011652 19817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:34.011760 19817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:34.019058 19817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:34.019171 19817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:34.019336 19817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:51:34.029996 19817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:45899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:34.031380 19817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:34.032964 19817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:34.046530 19822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:34.046581 19825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:34.046530 19823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:34.047518 19817 server_base.cc:1034] running on GCE node
I20260929 15:51:34.048179 19817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:34.049124 19817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:34.050345 19817 hybrid_clock.cc:648] HybridClock initialized: now 1790697094050284 us; error 46 us; skew 500 ppm
I20260929 15:51:34.050645 19817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:34 dist-test-slave-c51v krb5kdc[19711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697094, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:34.053893 19817 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:51:34.055473 19817 webserver.cc:460] Webserver started at http://127.15.196.193:41739/ using document root <none> and password file <none>
I20260929 15:51:34.056150 19817 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:34.056298 19817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:34.056612 19817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:34.059557 19817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "28ca14fb568c4dddbaffcb9078e0f89f"
format_stamp: "Formatted at 2026-09-29 15:51:34 on dist-test-slave-c51v"
server_key: "7a5a88ced648ab8ff21994db3c4bfc4d"
server_key_iv: "467db79102f36cacdb5fb090a08c90a4"
server_key_version: "encryptionkey@0"
I20260929 15:51:34.060314 19817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "28ca14fb568c4dddbaffcb9078e0f89f"
format_stamp: "Formatted at 2026-09-29 15:51:34 on dist-test-slave-c51v"
server_key: "7a5a88ced648ab8ff21994db3c4bfc4d"
server_key_iv: "467db79102f36cacdb5fb090a08c90a4"
server_key_version: "encryptionkey@0"
I20260929 15:51:34.065541 19817 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:51:34.069011 19832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:34.070264 19817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 15:51:34.070441 19817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "28ca14fb568c4dddbaffcb9078e0f89f"
format_stamp: "Formatted at 2026-09-29 15:51:34 on dist-test-slave-c51v"
server_key: "7a5a88ced648ab8ff21994db3c4bfc4d"
server_key_iv: "467db79102f36cacdb5fb090a08c90a4"
server_key_version: "encryptionkey@0"
I20260929 15:51:34.070652 19817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:34.094174 19817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:34.098393 19817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:34.098696 19817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:34.099717 19817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:34.101045 19817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:34.101147 19817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:34.101300 19817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:34.101373 19817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:34.137253 19817 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:44915
I20260929 15:51:34.137360 19944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:44915 every 8 connection(s)
I20260929 15:51:34.138705 19817 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:51:34.148871 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 19817
I20260929 15:51:34.149113 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:51:34.149739 16147 external_mini_cluster.cc:1428] Setting key 5070a2e4fc6281a5d833bef11661d667
Sep 29 15:51:34 dist-test-slave-c51v krb5kdc[19711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697094, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:34.159221 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:34.141460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:47387 (local address 127.15.196.254:34853)
0929 15:51:34.141920 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:34.141926 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:34.142968 (+ 1042us) server_negotiation.cc:407] Connection header received
0929 15:51:34.144907 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:34.144914 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:34.145013 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:34.145133 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:34.146663 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:34.147453 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:34.148988 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:34.149450 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:34.149583 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:34.153285 (+ 3702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:34.153290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:34.153294 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:34.153324 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:34.156123 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:34.156901 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:34.156906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:34.156916 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:34.157033 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:34.157459 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:34.157465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:34.157484 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:34.157756 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:34.157927 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:34.158640 (+ 713us) server_negotiation.cc:299] Negotiation successful
0929 15:51:34.158862 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":218,"threads_started":1}
I20260929 15:51:34.160229 19948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:34.142062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:34853 (local address 127.15.196.193:47387)
0929 15:51:34.142782 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:34.142826 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:34.144623 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:34.145379 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:34.145388 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:34.145906 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:34.146489 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:34.146506 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:34.147609 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:34.147614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:34.148834 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:34.148845 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:34.149686 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:34.149694 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:34.149979 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:34.150660 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:34.150711 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:34.153060 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:34.156270 (+ 3210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:34.156282 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:34.156300 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:34.156767 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:34.157142 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:34.157148 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:34.157151 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:34.157344 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:34.157896 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:34.157904 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:34.158250 (+ 346us) client_negotiation.cc:769] Sending connection context
0929 15:51:34.158524 (+ 274us) client_negotiation.cc:240] Negotiation successful
0929 15:51:34.159095 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":281,"threads_started":1}
I20260929 15:51:34.161969 19945 heartbeater.cc:347] Connected to a master server at 127.15.196.254:34853
I20260929 15:51:34.162295 19945 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:34.162992 19945 heartbeater.cc:511] Master 127.15.196.254:34853 requested a full tablet report, sending...
I20260929 15:51:34.165535 19756 ts_manager.cc:194] Registered new tserver with Master: 28ca14fb568c4dddbaffcb9078e0f89f (127.15.196.193:44915)
I20260929 15:51:34.167750 19756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:47387
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:51:34.194970 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34853
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:45899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:34.529064 19951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:34.529378 19951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:34.529541 19951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:34.537210 19951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:34.537369 19951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:34.537598 19951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:51:34.548136 19951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:45899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:34.549579 19951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:34.551102 19951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:34.565253 19959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:34.565255 19956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:34.565253 19957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:34.566242 19951 server_base.cc:1034] running on GCE node
I20260929 15:51:34.566883 19951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:34.567976 19951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:34.569206 19951 hybrid_clock.cc:648] HybridClock initialized: now 1790697094569173 us; error 35 us; skew 500 ppm
I20260929 15:51:34.569528 19951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:34 dist-test-slave-c51v krb5kdc[19711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697094, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:34.572764 19951 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:51:34.574281 19951 webserver.cc:460] Webserver started at http://127.15.196.194:36711/ using document root <none> and password file <none>
I20260929 15:51:34.574939 19951 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:34.575033 19951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:34.575358 19951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:34.578722 19951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "b8d347d72e35493ea08814ce294650e5"
format_stamp: "Formatted at 2026-09-29 15:51:34 on dist-test-slave-c51v"
server_key: "4c690af57357f1576eb3ede9c4922583"
server_key_iv: "a30947b3541b5e0ad644b2e786464de2"
server_key_version: "encryptionkey@0"
I20260929 15:51:34.579557 19951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "b8d347d72e35493ea08814ce294650e5"
format_stamp: "Formatted at 2026-09-29 15:51:34 on dist-test-slave-c51v"
server_key: "4c690af57357f1576eb3ede9c4922583"
server_key_iv: "a30947b3541b5e0ad644b2e786464de2"
server_key_version: "encryptionkey@0"
I20260929 15:51:34.584702 19951 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 15:51:34.588161 19966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:34.589491 19951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:51:34.589674 19951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "b8d347d72e35493ea08814ce294650e5"
format_stamp: "Formatted at 2026-09-29 15:51:34 on dist-test-slave-c51v"
server_key: "4c690af57357f1576eb3ede9c4922583"
server_key_iv: "a30947b3541b5e0ad644b2e786464de2"
server_key_version: "encryptionkey@0"
I20260929 15:51:34.589898 19951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:34.610285 19951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:34.614414 19951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:34.614715 19951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:34.615756 19951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:34.617121 19951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:34.617193 19951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:34.617360 19951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:34.617411 19951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:34.650982 19951 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:41069
I20260929 15:51:34.651028 20078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:41069 every 8 connection(s)
I20260929 15:51:34.652352 19951 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:51:34.654812 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 19951
I20260929 15:51:34.655031 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:51:34.655573 16147 external_mini_cluster.cc:1428] Setting key 664320df597ddb7d4499c7c3eeb80fa9
Sep 29 15:51:34 dist-test-slave-c51v krb5kdc[19711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697094, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
I20260929 15:51:34.674681 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:34.655586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:50829 (local address 127.15.196.254:34853)
0929 15:51:34.655819 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:34.655826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:34.657561 (+ 1735us) server_negotiation.cc:407] Connection header received
0929 15:51:34.659692 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:34.659697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:34.659800 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:34.659931 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:34.661711 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:34.662495 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:34.664253 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:34.664723 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:34.664880 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:34.668806 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:34.668812 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:34.668815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:34.668849 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:34.671511 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:34.672275 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:34.672280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:34.672290 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:34.672409 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:34.672841 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:34.672846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:34.672849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:34.673176 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:34.673329 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:34.674076 (+ 747us) server_negotiation.cc:299] Negotiation successful
0929 15:51:34.674310 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
I20260929 15:51:34.675995 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:34.656378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:34853 (local address 127.15.196.194:50829)
0929 15:51:34.657370 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:34.657420 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:34.659398 (+ 1978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:34.660198 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:34.660207 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:34.660767 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:34.661530 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:34.661550 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:34.662670 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:34.662675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:34.664088 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:34.664101 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:34.664981 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:34.664988 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:34.665233 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:34.665977 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:34.666006 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:34.668551 (+ 2545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:34.671665 (+ 3114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:34.671675 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:34.671695 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:34.672140 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:34.672526 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:34.672531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:34.672534 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:34.672723 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:34.673303 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:34.673312 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:34.673692 (+ 380us) client_negotiation.cc:769] Sending connection context
0929 15:51:34.673999 (+ 307us) client_negotiation.cc:240] Negotiation successful
0929 15:51:34.674601 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":423,"threads_started":1}
I20260929 15:51:34.677752 20079 heartbeater.cc:347] Connected to a master server at 127.15.196.254:34853
I20260929 15:51:34.678115 20079 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:34.678962 20079 heartbeater.cc:511] Master 127.15.196.254:34853 requested a full tablet report, sending...
I20260929 15:51:34.680649 19756 ts_manager.cc:194] Registered new tserver with Master: b8d347d72e35493ea08814ce294650e5 (127.15.196.194:41069)
I20260929 15:51:34.681807 19756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:50829
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:51:34.702828 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34853
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:45899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:35.032404 20084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:35.032792 20084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:35.032967 20084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:35.040874 20084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:35.041054 20084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:35.041260 20084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:51:35.051561 20084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:45899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:35.052999 20084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:35.054386 20084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:35.068511 20089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:35.068574 20090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:35.068382 20092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:35.069796 20084 server_base.cc:1034] running on GCE node
I20260929 15:51:35.070559 20084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:35.071681 20084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:35.073025 20084 hybrid_clock.cc:648] HybridClock initialized: now 1790697095072963 us; error 64 us; skew 500 ppm
I20260929 15:51:35.073407 20084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:35 dist-test-slave-c51v krb5kdc[19711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697095, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:35.077258 20084 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:51:35.078990 20084 webserver.cc:460] Webserver started at http://127.15.196.195:41315/ using document root <none> and password file <none>
I20260929 15:51:35.079829 20084 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:35.079995 20084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:35.080333 20084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:35.083819 20084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "804ff77c06ff4355b4694b9e785a5db5"
format_stamp: "Formatted at 2026-09-29 15:51:35 on dist-test-slave-c51v"
server_key: "5878c78a198f69635fd8238358a2e43a"
server_key_iv: "24e6147b099988dfed307987aaf4122c"
server_key_version: "encryptionkey@0"
I20260929 15:51:35.084685 20084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "804ff77c06ff4355b4694b9e785a5db5"
format_stamp: "Formatted at 2026-09-29 15:51:35 on dist-test-slave-c51v"
server_key: "5878c78a198f69635fd8238358a2e43a"
server_key_iv: "24e6147b099988dfed307987aaf4122c"
server_key_version: "encryptionkey@0"
I20260929 15:51:35.090240 20084 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:51:35.094002 20099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:35.095592 20084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:35.095765 20084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "804ff77c06ff4355b4694b9e785a5db5"
format_stamp: "Formatted at 2026-09-29 15:51:35 on dist-test-slave-c51v"
server_key: "5878c78a198f69635fd8238358a2e43a"
server_key_iv: "24e6147b099988dfed307987aaf4122c"
server_key_version: "encryptionkey@0"
I20260929 15:51:35.095970 20084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:35.116091 20084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:35.120800 20084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:35.121104 20084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:35.122247 20084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:35.123782 20084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:35.123883 20084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:35.124040 20084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:35.124109 20084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:35.160765 20084 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:41623
I20260929 15:51:35.160820 20211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:41623 every 8 connection(s)
I20260929 15:51:35.162062 20084 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:51:35.162338 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 20084
I20260929 15:51:35.162534 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:51:35.163107 16147 external_mini_cluster.cc:1428] Setting key 7252eda033a5434975f209a97288ce10
I20260929 15:51:35.171175 19945 heartbeater.cc:503] Master 127.15.196.254:34853 was elected leader, sending a full tablet report...
Sep 29 15:51:35 dist-test-slave-c51v krb5kdc[19711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697095, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:35.182850 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:35.164906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:34319 (local address 127.15.196.254:34853)
0929 15:51:35.165097 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:35.165102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:35.166544 (+ 1442us) server_negotiation.cc:407] Connection header received
0929 15:51:35.168469 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:35.168474 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:35.168597 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:35.168708 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:35.170206 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:35.171002 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:35.172627 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:35.173054 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:35.173151 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:35.176644 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:35.176649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:35.176652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:35.176682 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:35.179789 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:35.180524 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:35.180530 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:35.180533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:35.180665 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:35.181095 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:35.181105 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:35.181107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:35.181367 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:35.181526 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:35.182268 (+ 742us) server_negotiation.cc:299] Negotiation successful
0929 15:51:35.182488 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 15:51:35.183903 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:35.165531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:34853 (local address 127.15.196.195:34319)
0929 15:51:35.166381 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:35.166427 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:35.168201 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:35.168954 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:35.168963 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:35.169484 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:35.170041 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:35.170058 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:35.171171 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:35.171175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:35.172516 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:35.172528 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:35.173244 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:35.173249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:35.173457 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:35.174157 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:35.174196 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:35.176409 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:35.179927 (+ 3518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:35.179937 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:35.179954 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:35.180389 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:35.180775 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:35.180780 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:35.180784 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:35.180975 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:35.181502 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:35.181510 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:35.181881 (+ 371us) client_negotiation.cc:769] Sending connection context
0929 15:51:35.182159 (+ 278us) client_negotiation.cc:240] Negotiation successful
0929 15:51:35.182722 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":314,"threads_started":1}
I20260929 15:51:35.185210 20212 heartbeater.cc:347] Connected to a master server at 127.15.196.254:34853
I20260929 15:51:35.185531 20212 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:35.186252 20212 heartbeater.cc:511] Master 127.15.196.254:34853 requested a full tablet report, sending...
I20260929 15:51:35.187682 19756 ts_manager.cc:194] Registered new tserver with Master: 804ff77c06ff4355b4694b9e785a5db5 (127.15.196.195:41623)
I20260929 15:51:35.188479 19756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:34319
I20260929 15:51:35.190989 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 15:51:35 dist-test-slave-c51v krb5kdc[19711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 15:51:35 dist-test-slave-c51v krb5kdc[19711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:51:35.239442 20219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:35.224193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53344 (local address 127.15.196.193:44915)
0929 15:51:35.224924 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:35.224935 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:35.224970 (+ 35us) server_negotiation.cc:407] Connection header received
0929 15:51:35.225048 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:35.225054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:35.225269 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:35.225469 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:35.226912 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:35.228039 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:35.229278 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:35.229841 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:35.229982 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:35.233996 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:35.234007 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:35.234023 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:35.234062 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:35.237044 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:35.237673 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:35.237680 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:35.237683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:35.237806 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:35.238121 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:35.238124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:35.238126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:35.238441 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:35.238656 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:35.238873 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 15:51:35.239078 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":369,"threads_started":1}
Sep 29 15:51:35 dist-test-slave-c51v krb5kdc[19711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:35.256433 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:35.243679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54396 (local address 127.15.196.254:34853)
0929 15:51:35.243899 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:35.243904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:35.244001 (+ 97us) server_negotiation.cc:407] Connection header received
0929 15:51:35.244132 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:35.244136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:35.244196 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:35.244275 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:35.245431 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:35.246143 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:35.247355 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:35.247731 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:35.247823 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:35.251513 (+ 3690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:35.251518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:35.251521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:35.251552 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:35.254087 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:35.254653 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:35.254657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:35.254660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:35.254775 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:35.255068 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:35.255071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:35.255073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:35.255458 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:35.255649 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:35.255914 (+ 265us) server_negotiation.cc:299] Negotiation successful
0929 15:51:35.256073 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260929 15:51:35.257061 19756 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54396
I20260929 15:51:35.258109 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 19817
I20260929 15:51:35.270714 20219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:35.259045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53360 (local address 127.15.196.193:44915)
0929 15:51:35.259258 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:35.259265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:35.259479 (+ 214us) server_negotiation.cc:407] Connection header received
0929 15:51:35.259663 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:35.259669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:35.259771 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:35.259876 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:35.261050 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:35.261756 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:35.262950 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:35.263451 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:35.263582 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:35.265747 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:35.265753 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:35.265758 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:35.265799 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:35.268217 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:35.268847 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:35.268854 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:35.268857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:35.268990 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:35.269352 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:35.269358 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:35.269362 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:35.269624 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:35.269864 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:35.270176 (+ 312us) server_negotiation.cc:299] Negotiation successful
0929 15:51:35.270396 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 15:51:35.271402 19940 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:35.355098 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 19817
I20260929 15:51:35.371073 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 19951
Sep 29 15:51:35 dist-test-slave-c51v krb5kdc[19711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:51:35.386402 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:35.372035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40242 (local address 127.15.196.194:41069)
0929 15:51:35.372563 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:35.372571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:35.372594 (+ 23us) server_negotiation.cc:407] Connection header received
0929 15:51:35.372653 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:35.372659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:35.372869 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:35.373002 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:35.374144 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:35.375133 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:35.376441 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:35.376986 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:35.377081 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:35.381030 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:35.381041 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:35.381055 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:35.381092 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:35.383780 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:35.384361 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:35.384367 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:35.384370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:35.384502 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:35.384810 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:35.384813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:35.384814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:35.385127 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:35.385342 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:35.385665 (+ 323us) server_negotiation.cc:299] Negotiation successful
0929 15:51:35.386045 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":253,"threads_started":1}
I20260929 15:51:35.387882 20074 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:35.469494 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 19951
I20260929 15:51:35.485960 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 20084
Sep 29 15:51:35 dist-test-slave-c51v krb5kdc[19711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:51:35.502266 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:35.486919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49964 (local address 127.15.196.195:41623)
0929 15:51:35.487588 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:35.487595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:35.487622 (+ 27us) server_negotiation.cc:407] Connection header received
0929 15:51:35.487679 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:35.487686 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:35.487913 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:35.488042 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:35.489232 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:35.490245 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:35.491532 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:35.492085 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:35.492180 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:35.496544 (+ 4364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:35.496556 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:35.496571 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:35.496610 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:35.499695 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:35.500417 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:35.500424 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:35.500426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:35.500543 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:35.500884 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:35.500888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:35.500889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:35.501230 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:35.501458 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:35.501682 (+ 224us) server_negotiation.cc:299] Negotiation successful
0929 15:51:35.501875 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":367,"threads_started":1}
I20260929 15:51:35.503716 20207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:35.589715 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 20084
I20260929 15:51:35.605723 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 19724
I20260929 15:51:35.606974 19786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:35.710651 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 19724
2026-09-29T15:51:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3959 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:51:35 dist-test-slave-c51v krb5kdc[20231](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:51:35 dist-test-slave-c51v krb5kdc[20231](info): set up 2 sockets
Sep 29 15:51:35 dist-test-slave-c51v krb5kdc[20231](info): commencing operation
krb5kdc: starting...
W20260929 15:51:36.793814 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.002s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:51:36 dist-test-slave-c51v krb5kdc[20231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:51:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:51:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:51:36.999143 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:46575
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:34761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:46575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:37.326479 20244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:37.326784 20244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:37.326967 20244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:37.334271 20244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:37.334388 20244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:37.334493 20244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:37.334591 20244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:37.334666 20244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:51:37.344728 20244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:34761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:46575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:46575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:37.346127 20244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:37.347492 20244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:37.359129 20249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:37.359163 20250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:37.359095 20252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:37.360134 20244 server_base.cc:1034] running on GCE node
I20260929 15:51:37.360944 20244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:37.362391 20244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:37.363623 20244 hybrid_clock.cc:648] HybridClock initialized: now 1790697097363579 us; error 41 us; skew 500 ppm
I20260929 15:51:37.363986 20244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:37 dist-test-slave-c51v krb5kdc[20231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697097, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:37.372354 20244 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:51:37.374027 20244 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:37.374162 20244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:37.374503 20244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:37.377645 20244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "13abf444ff7c45e5a4454c459083ebff"
format_stamp: "Formatted at 2026-09-29 15:51:37 on dist-test-slave-c51v"
server_key: "1e98f947e504baebe89557b41e0b455d"
server_key_iv: "57c7bd4915cc22fba006b315507da373"
server_key_version: "encryptionkey@0"
I20260929 15:51:37.378448 20244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "13abf444ff7c45e5a4454c459083ebff"
format_stamp: "Formatted at 2026-09-29 15:51:37 on dist-test-slave-c51v"
server_key: "1e98f947e504baebe89557b41e0b455d"
server_key_iv: "57c7bd4915cc22fba006b315507da373"
server_key_version: "encryptionkey@0"
I20260929 15:51:37.383615 20244 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260929 15:51:37.387234 20258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:37.388660 20244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:37.388847 20244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "13abf444ff7c45e5a4454c459083ebff"
format_stamp: "Formatted at 2026-09-29 15:51:37 on dist-test-slave-c51v"
server_key: "1e98f947e504baebe89557b41e0b455d"
server_key_iv: "57c7bd4915cc22fba006b315507da373"
server_key_version: "encryptionkey@0"
I20260929 15:51:37.389070 20244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:37.420769 20244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:37.424823 20244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:37.425100 20244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:37.443432 20244 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:46575
I20260929 15:51:37.443485 20309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:46575 every 8 connection(s)
I20260929 15:51:37.444350 20244 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:51:37.448097 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 20244
I20260929 15:51:37.448169 20310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:37.448340 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:51:37.449127 16147 external_mini_cluster.cc:1428] Setting key 34b2d36dcf2e90c1c2bf7d9e34216f77
I20260929 15:51:37.462292 20310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff: Bootstrap starting.
Sep 29 15:51:37 dist-test-slave-c51v krb5kdc[20231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:37.466017 20310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:37.467326 20310 log.cc:824] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:37.470147 20310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff: No bootstrap required, opened a new log
I20260929 15:51:37.472878 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:37.451223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44652 (local address 127.15.196.254:46575)
0929 15:51:37.452130 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:37.452143 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:37.452181 (+ 38us) server_negotiation.cc:407] Connection header received
0929 15:51:37.454395 (+ 2214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:37.454426 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:37.454825 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:37.455162 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:37.456969 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:37.458247 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:37.460130 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:37.460778 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:37.460910 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:37.465612 (+ 4702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:37.465621 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:37.465635 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:37.465674 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:37.468798 (+ 3124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:37.469483 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:37.469490 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:37.469496 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:37.469649 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:37.470040 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:37.470049 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:37.470051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:37.470648 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:37.470964 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:37.471424 (+ 460us) server_negotiation.cc:299] Negotiation successful
0929 15:51:37.471844 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":413,"threads_started":1}
I20260929 15:51:37.478055 20310 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13abf444ff7c45e5a4454c459083ebff" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46575 } }
I20260929 15:51:37.478420 20310 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:37.478528 20310 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13abf444ff7c45e5a4454c459083ebff, State: Initialized, Role: FOLLOWER
I20260929 15:51:37.479110 20310 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13abf444ff7c45e5a4454c459083ebff" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46575 } }
I20260929 15:51:37.479399 20310 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:51:37.479568 20310 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:51:37.479743 20310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:37.481969 20310 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13abf444ff7c45e5a4454c459083ebff" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46575 } }
I20260929 15:51:37.482462 20310 leader_election.cc:302] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13abf444ff7c45e5a4454c459083ebff; no voters:
I20260929 15:51:37.483064 20310 leader_election.cc:288] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:51:37.483254 20315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:37.484071 20315 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [term 1 LEADER]: Becoming Leader. State: Replica: 13abf444ff7c45e5a4454c459083ebff, State: Running, Role: LEADER
I20260929 15:51:37.484647 20315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13abf444ff7c45e5a4454c459083ebff" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46575 } }
I20260929 15:51:37.485234 20310 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:37.487746 20316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13abf444ff7c45e5a4454c459083ebff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13abf444ff7c45e5a4454c459083ebff" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46575 } } }
I20260929 15:51:37.488184 20316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:37.488047 20317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13abf444ff7c45e5a4454c459083ebff. Latest consensus state: current_term: 1 leader_uuid: "13abf444ff7c45e5a4454c459083ebff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13abf444ff7c45e5a4454c459083ebff" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46575 } } }
I20260929 15:51:37.488379 20317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:37.490170 20320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:37.505326 20320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:37.515249 20320 catalog_manager.cc:1347] Generated new cluster ID: 47189772eadb4d5f82732ec695c23467
I20260929 15:51:37.515391 20320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:37.541023 20320 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:51:37.542619 20320 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:37.548576 20320 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff: Generated new TSK 0
I20260929 15:51:37.549397 20320 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:51:37.603138 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46575
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:34761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:37.937136 20336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:37.937457 20336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:37.937645 20336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:37.945200 20336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:37.945358 20336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:37.945533 20336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:51:37.955997 20336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:34761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46575
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:37.957365 20336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:37.958679 20336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:37.974119 20341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:37.974393 20344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:37.974756 20336 server_base.cc:1034] running on GCE node
W20260929 15:51:37.974980 20342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:37.975705 20336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:37.976745 20336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:37.977977 20336 hybrid_clock.cc:648] HybridClock initialized: now 1790697097977911 us; error 68 us; skew 500 ppm
I20260929 15:51:37.978303 20336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:37 dist-test-slave-c51v krb5kdc[20231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697097, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:37.981698 20336 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:51:37.983033 20336 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:37.983161 20336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:37.983562 20336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:37.986749 20336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "c98d8a6dccfe4f8b8e62f1b922610a64"
format_stamp: "Formatted at 2026-09-29 15:51:37 on dist-test-slave-c51v"
server_key: "eaf9a5c59f6fba4fe5f0cccb0a01a065"
server_key_iv: "1a2f238db6dce644aabf8fc41680a39b"
server_key_version: "encryptionkey@0"
I20260929 15:51:37.987675 20336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "c98d8a6dccfe4f8b8e62f1b922610a64"
format_stamp: "Formatted at 2026-09-29 15:51:37 on dist-test-slave-c51v"
server_key: "eaf9a5c59f6fba4fe5f0cccb0a01a065"
server_key_iv: "1a2f238db6dce644aabf8fc41680a39b"
server_key_version: "encryptionkey@0"
I20260929 15:51:37.993342 20336 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:51:37.997071 20350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:37.998551 20336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 15:51:37.998729 20336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "c98d8a6dccfe4f8b8e62f1b922610a64"
format_stamp: "Formatted at 2026-09-29 15:51:37 on dist-test-slave-c51v"
server_key: "eaf9a5c59f6fba4fe5f0cccb0a01a065"
server_key_iv: "1a2f238db6dce644aabf8fc41680a39b"
server_key_version: "encryptionkey@0"
I20260929 15:51:37.998963 20336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:38.023917 20336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:38.028426 20336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:38.028719 20336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:38.029812 20336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:38.031371 20336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:38.031484 20336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:38.031608 20336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:38.031708 20336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:38.070912 20336 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:34967
I20260929 15:51:38.070940 20462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:34967 every 8 connection(s)
I20260929 15:51:38.071885 20336 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:51:38.074537 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 20336
I20260929 15:51:38.074775 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:51:38.075523 16147 external_mini_cluster.cc:1428] Setting key c0d38fefb5459065cfdae6e1202b8a4f
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
Sep 29 15:51:38 dist-test-slave-c51v krb5kdc[20231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697097, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:38.114576 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:38.075544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:51447 (local address 127.15.196.254:46575)
0929 15:51:38.089477 (+ 13933us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:38.089481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:38.089889 (+ 408us) server_negotiation.cc:407] Connection header received
0929 15:51:38.093465 (+ 3576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:38.093486 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:38.093585 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:38.093714 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:38.096888 (+ 3174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:38.098194 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:38.101033 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:38.101862 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:38.102022 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:38.107674 (+ 5652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:38.107684 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:38.107691 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:38.107739 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:38.111488 (+ 3749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:38.112244 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:38.112251 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:38.112259 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:38.112385 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:38.112812 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:38.112822 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:38.112824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:38.113082 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:38.113236 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:38.113947 (+ 711us) server_negotiation.cc:299] Negotiation successful
0929 15:51:38.114162 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13788,"thread_start_us":13679,"threads_started":1}
I20260929 15:51:38.115723 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:38.075409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:46575 (local address 127.15.196.193:51447)
0929 15:51:38.089663 (+ 14254us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:38.089733 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:38.092968 (+ 3235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:38.094708 (+ 1740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:38.094724 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:38.095693 (+ 969us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:38.096668 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:38.096697 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:38.098426 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:38.098435 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:38.100832 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:38.100849 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:38.102191 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:38.102201 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:38.102507 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:38.103428 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:38.103450 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:38.107383 (+ 3933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:38.111631 (+ 4248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:38.111641 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:38.111660 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:38.112109 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:38.112500 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:38.112505 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:38.112509 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:38.112692 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:38.113209 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:38.113217 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:38.113575 (+ 358us) client_negotiation.cc:769] Sending connection context
0929 15:51:38.113846 (+ 271us) client_negotiation.cc:240] Negotiation successful
0929 15:51:38.114409 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13796,"thread_start_us":17933,"threads_started":1}
I20260929 15:51:38.117316 20463 heartbeater.cc:347] Connected to a master server at 127.15.196.254:46575
I20260929 15:51:38.117658 20463 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:38.118325 20463 heartbeater.cc:511] Master 127.15.196.254:46575 requested a full tablet report, sending...
I20260929 15:51:38.120507 20275 ts_manager.cc:194] Registered new tserver with Master: c98d8a6dccfe4f8b8e62f1b922610a64 (127.15.196.193:34967)
I20260929 15:51:38.122162 20275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:51447
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:51:38.138350 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46575
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:34761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:38.498696 20469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:38.499003 20469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:38.499109 20469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:38.506903 20469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:38.507027 20469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:38.507164 20469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:51:38.517258 20469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:34761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46575
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:38.518570 20469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:38.520030 20469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:38.533165 20474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:38.533129 20475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:38.533252 20477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:38.534592 20469 server_base.cc:1034] running on GCE node
I20260929 15:51:38.535367 20469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:38.536407 20469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:38.537650 20469 hybrid_clock.cc:648] HybridClock initialized: now 1790697098537609 us; error 42 us; skew 500 ppm
I20260929 15:51:38.538025 20469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:38 dist-test-slave-c51v krb5kdc[20231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697098, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:38.541458 20469 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:51:38.542644 20469 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:38.542780 20469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:38.543119 20469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:38.546191 20469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "8ef8f0d6d48d411f9fca75984345e72b"
format_stamp: "Formatted at 2026-09-29 15:51:38 on dist-test-slave-c51v"
server_key: "f591a8ceae11b4a63316ee11f8a9c3ee"
server_key_iv: "b712d266485f9c9ad1166512185ea83d"
server_key_version: "encryptionkey@0"
I20260929 15:51:38.547062 20469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "8ef8f0d6d48d411f9fca75984345e72b"
format_stamp: "Formatted at 2026-09-29 15:51:38 on dist-test-slave-c51v"
server_key: "f591a8ceae11b4a63316ee11f8a9c3ee"
server_key_iv: "b712d266485f9c9ad1166512185ea83d"
server_key_version: "encryptionkey@0"
I20260929 15:51:38.552613 20469 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 15:51:38.556202 20483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:38.557653 20469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260929 15:51:38.557843 20469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "8ef8f0d6d48d411f9fca75984345e72b"
format_stamp: "Formatted at 2026-09-29 15:51:38 on dist-test-slave-c51v"
server_key: "f591a8ceae11b4a63316ee11f8a9c3ee"
server_key_iv: "b712d266485f9c9ad1166512185ea83d"
server_key_version: "encryptionkey@0"
I20260929 15:51:38.558079 20469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:38.578868 20469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:38.583621 20469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:38.583920 20469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:38.584923 20469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:38.586431 20469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:38.586550 20469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:38.586695 20469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:38.586782 20469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:38.620571 20469 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:33411
I20260929 15:51:38.620622 20595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:33411 every 8 connection(s)
I20260929 15:51:38.621457 20469 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:51:38.622922 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 20469
I20260929 15:51:38.623119 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:51:38.623665 16147 external_mini_cluster.cc:1428] Setting key dfbb82e4843b9e8c193cc43bd283e9c4
Sep 29 15:51:38 dist-test-slave-c51v krb5kdc[20231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697098, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
I20260929 15:51:38.641402 20598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:38.624069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:33345 (local address 127.15.196.254:46575)
0929 15:51:38.624529 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:38.624535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:38.625902 (+ 1367us) server_negotiation.cc:407] Connection header received
0929 15:51:38.627494 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:38.627499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:38.627580 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:38.627705 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:38.629156 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:38.629868 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:38.631451 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:38.631870 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:38.632005 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:38.635367 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:38.635372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:38.635375 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:38.635412 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:38.638052 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:38.638918 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:38.638925 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:38.638929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:38.639061 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:38.639583 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:38.639589 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:38.639592 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:38.639881 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:38.640074 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:38.640787 (+ 713us) server_negotiation.cc:299] Negotiation successful
0929 15:51:38.641017 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":219,"threads_started":1}
I20260929 15:51:38.642386 20599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:38.624670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:46575 (local address 127.15.196.194:33345)
0929 15:51:38.625432 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:38.625484 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:38.627180 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:38.627941 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:38.627950 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:38.628443 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:38.628992 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:38.629008 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:38.630014 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:38.630019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:38.631234 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:38.631246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:38.632089 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:38.632096 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:38.632324 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:38.633025 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:38.633047 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:38.635112 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:38.638234 (+ 3122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:38.638244 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:38.638263 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:38.638703 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:38.639179 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:38.639184 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:38.639187 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:38.639469 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:38.640043 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:38.640051 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:38.640415 (+ 364us) client_negotiation.cc:769] Sending connection context
0929 15:51:38.640685 (+ 270us) client_negotiation.cc:240] Negotiation successful
0929 15:51:38.641250 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":296,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
I20260929 15:51:38.643975 20596 heartbeater.cc:347] Connected to a master server at 127.15.196.254:46575
I20260929 15:51:38.644243 20596 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:38.644848 20596 heartbeater.cc:511] Master 127.15.196.254:46575 requested a full tablet report, sending...
I20260929 15:51:38.646334 20275 ts_manager.cc:194] Registered new tserver with Master: 8ef8f0d6d48d411f9fca75984345e72b (127.15.196.194:33411)
I20260929 15:51:38.647153 20275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:33345
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:51:38.667152 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46575
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:34761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:38.995886 20602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:38.996253 20602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:38.996407 20602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:39.003934 20602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:39.004043 20602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:39.004249 20602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:51:39.014219 20602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:34761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46575
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:39.015616 20602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:39.016995 20602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:39.030387 20607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:39.030390 20610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:39.030329 20608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:39.031543 20602 server_base.cc:1034] running on GCE node
I20260929 15:51:39.032166 20602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:39.033121 20602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:39.034390 20602 hybrid_clock.cc:648] HybridClock initialized: now 1790697099034313 us; error 73 us; skew 500 ppm
I20260929 15:51:39.034749 20602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:39 dist-test-slave-c51v krb5kdc[20231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697099, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:39.038406 20602 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:51:39.039649 20602 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:39.039783 20602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:39.040112 20602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:39.043095 20602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "76eef21033994522bc51a6de63b57904"
format_stamp: "Formatted at 2026-09-29 15:51:39 on dist-test-slave-c51v"
server_key: "bf0b5ac5ec6adeaa89ddac0f0888ba20"
server_key_iv: "85aac28926feb22b7c115a08a90ec1e8"
server_key_version: "encryptionkey@0"
I20260929 15:51:39.044006 20602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "76eef21033994522bc51a6de63b57904"
format_stamp: "Formatted at 2026-09-29 15:51:39 on dist-test-slave-c51v"
server_key: "bf0b5ac5ec6adeaa89ddac0f0888ba20"
server_key_iv: "85aac28926feb22b7c115a08a90ec1e8"
server_key_version: "encryptionkey@0"
I20260929 15:51:39.049808 20602 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260929 15:51:39.054250 20616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:39.055755 20602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:39.055945 20602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "76eef21033994522bc51a6de63b57904"
format_stamp: "Formatted at 2026-09-29 15:51:39 on dist-test-slave-c51v"
server_key: "bf0b5ac5ec6adeaa89ddac0f0888ba20"
server_key_iv: "85aac28926feb22b7c115a08a90ec1e8"
server_key_version: "encryptionkey@0"
I20260929 15:51:39.056160 20602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:39.098470 20602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:39.102797 20602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:39.103235 20602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:39.104682 20602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:39.106225 20602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:39.106387 20602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:39.106534 20602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:39.106639 20602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:39.125211 20463 heartbeater.cc:503] Master 127.15.196.254:46575 was elected leader, sending a full tablet report...
I20260929 15:51:39.143507 20602 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:46581
I20260929 15:51:39.143573 20728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:46581 every 8 connection(s)
I20260929 15:51:39.144831 20602 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:51:39.149000 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 20602
I20260929 15:51:39.149207 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:51:39.149639 16147 external_mini_cluster.cc:1428] Setting key 952170efc640f480a3f7862522a2900a
Sep 29 15:51:39 dist-test-slave-c51v krb5kdc[20231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697099, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:39.166172 20731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.148255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:52843 (local address 127.15.196.254:46575)
0929 15:51:39.148748 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:39.148757 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:39.149786 (+ 1029us) server_negotiation.cc:407] Connection header received
0929 15:51:39.152206 (+ 2420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:39.152210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:39.152272 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:39.152358 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:39.153879 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.154632 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.156180 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.156573 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.156668 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.160005 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:39.160010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:39.160014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:39.160045 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:39.162832 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:39.163732 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:39.163736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:39.163739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:39.163909 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:39.164359 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:39.164364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:39.164367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:39.164713 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:39.164878 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:39.165577 (+ 699us) server_negotiation.cc:299] Negotiation successful
0929 15:51:39.165791 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":238,"threads_started":1}
I20260929 15:51:39.167197 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.148721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:46575 (local address 127.15.196.195:52843)
0929 15:51:39.149543 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:39.149598 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:39.151936 (+ 2338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:39.152630 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:39.152642 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:39.153179 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:39.153736 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:39.153752 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.154751 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.154754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:39.156053 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:39.156064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.156781 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.156788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:39.157010 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.157625 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:39.157646 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:39.159824 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:39.163006 (+ 3182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:39.163015 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:39.163038 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:39.163612 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:39.164044 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:39.164050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:39.164054 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:39.164242 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:39.164848 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:39.164856 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:39.165215 (+ 359us) client_negotiation.cc:769] Sending connection context
0929 15:51:39.165487 (+ 272us) client_negotiation.cc:240] Negotiation successful
0929 15:51:39.166021 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":352,"threads_started":1}
I20260929 15:51:39.168494 20729 heartbeater.cc:347] Connected to a master server at 127.15.196.254:46575
I20260929 15:51:39.168812 20729 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:39.169474 20729 heartbeater.cc:511] Master 127.15.196.254:46575 requested a full tablet report, sending...
I20260929 15:51:39.170831 20275 ts_manager.cc:194] Registered new tserver with Master: 76eef21033994522bc51a6de63b57904 (127.15.196.195:46581)
I20260929 15:51:39.171653 20275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:52843
I20260929 15:51:39.177114 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:51:39.194339 20731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.181641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44664 (local address 127.15.196.254:46575)
0929 15:51:39.181816 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:39.181821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:39.182420 (+ 599us) server_negotiation.cc:407] Connection header received
0929 15:51:39.182549 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:39.182552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:39.182627 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:39.182711 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:39.184126 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.184828 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.186161 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.186670 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.186770 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.189080 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:39.189085 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:39.189088 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:39.189116 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:39.191773 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:39.192518 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:39.192522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:39.192524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:39.192649 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:39.193078 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:39.193081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:39.193083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:39.193428 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:39.193610 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:39.193940 (+ 330us) server_negotiation.cc:299] Negotiation successful
0929 15:51:39.194103 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 15:51:39.198730 20275 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:51:39.205786 20275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:51:39.242040 20743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.217775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56738 (local address 127.15.196.194:33411)
0929 15:51:39.232991 (+ 15216us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:39.232998 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:39.233018 (+ 20us) server_negotiation.cc:407] Connection header received
0929 15:51:39.233145 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:39.233152 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:39.233361 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:39.234278 (+ 917us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:39.234497 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.235952 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.238742 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.240231 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.240408 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.240601 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:39.241505 (+ 904us) server_negotiation.cc:299] Negotiation successful
0929 15:51:39.241677 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15072,"thread_start_us":14909,"threads_started":1}
I20260929 15:51:39.242040 20745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.218324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46342 (local address 127.15.196.195:46581)
0929 15:51:39.232480 (+ 14156us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:39.232490 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:39.232524 (+ 34us) server_negotiation.cc:407] Connection header received
0929 15:51:39.232635 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:39.232645 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:39.232920 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:39.233122 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:39.233836 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.235558 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.238938 (+ 3380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.240332 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.240503 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.240651 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:39.241505 (+ 854us) server_negotiation.cc:299] Negotiation successful
0929 15:51:39.241677 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13979,"thread_start_us":13819,"threads_started":1}
I20260929 15:51:39.242906 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.218008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:46581 (local address 127.0.0.1:46342)
0929 15:51:39.232231 (+ 14223us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:39.232260 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:39.232405 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:39.233157 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:39.233164 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:39.233194 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:39.233576 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:39.233589 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.235708 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.235714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:39.238755 (+ 3041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:39.238770 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.240788 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.240793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:39.241132 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.241289 (+ 157us) client_negotiation.cc:769] Sending connection context
0929 15:51:39.242254 (+ 965us) client_negotiation.cc:240] Negotiation successful
0929 15:51:39.242466 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14061,"mutex_wait_us":690,"thread_start_us":13130,"threads_started":1}
I20260929 15:51:39.242934 20744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.217652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:33411 (local address 127.0.0.1:56738)
0929 15:51:39.230583 (+ 12931us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:39.230656 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:39.230904 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:39.233601 (+ 2697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:39.233609 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:39.233639 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:39.234065 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:39.234080 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.236097 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.236103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:39.238595 (+ 2492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:39.238607 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.240757 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.240765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:39.241129 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.241290 (+ 161us) client_negotiation.cc:769] Sending connection context
0929 15:51:39.242335 (+ 1045us) client_negotiation.cc:240] Negotiation successful
0929 15:51:39.242465 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12555,"mutex_wait_us":159,"thread_start_us":14709,"threads_started":1}
I20260929 15:51:39.246548 20531 tablet_service.cc:1467] Processing CreateTablet for tablet 1b211359d4374d7c9ee0c244683e419e (DEFAULT_TABLE table=test-table [id=951ee3668fe94d51aba7961b573c261d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:39.246559 20664 tablet_service.cc:1467] Processing CreateTablet for tablet 1b211359d4374d7c9ee0c244683e419e (DEFAULT_TABLE table=test-table [id=951ee3668fe94d51aba7961b573c261d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:39.247785 20531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b211359d4374d7c9ee0c244683e419e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:39.247786 20664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b211359d4374d7c9ee0c244683e419e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:39.249540 20746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.233096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34992 (local address 127.15.196.193:34967)
0929 15:51:39.237089 (+ 3993us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:39.237097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:39.237701 (+ 604us) server_negotiation.cc:407] Connection header received
0929 15:51:39.237982 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:39.237992 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:39.238193 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:39.238341 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:39.239171 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.240190 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.242679 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.243863 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.244323 (+ 460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.244490 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:39.248674 (+ 4184us) server_negotiation.cc:299] Negotiation successful
0929 15:51:39.249201 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3791,"thread_start_us":3632,"threads_started":1}
I20260929 15:51:39.250335 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.233570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34967 (local address 127.0.0.1:34992)
0929 15:51:39.237597 (+ 4027us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:39.237618 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:39.237798 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:39.238533 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:39.238537 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:39.238564 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:39.239025 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:39.239033 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.240327 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.240335 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:39.242513 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:39.242529 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.248077 (+ 5548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.248084 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:39.248259 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.248305 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 15:51:39.249800 (+ 1495us) client_negotiation.cc:240] Negotiation successful
0929 15:51:39.249955 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3879,"mutex_wait_us":2,"thread_start_us":3729,"threads_started":1}
I20260929 15:51:39.254384 20398 tablet_service.cc:1467] Processing CreateTablet for tablet 1b211359d4374d7c9ee0c244683e419e (DEFAULT_TABLE table=test-table [id=951ee3668fe94d51aba7961b573c261d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:39.255897 20398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b211359d4374d7c9ee0c244683e419e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:39.261405 20748 tablet_bootstrap.cc:492] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b: Bootstrap starting.
I20260929 15:51:39.263581 20749 tablet_bootstrap.cc:492] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904: Bootstrap starting.
I20260929 15:51:39.265471 20748 tablet_bootstrap.cc:654] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:39.266793 20748 log.cc:824] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:39.267043 20749 tablet_bootstrap.cc:654] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:39.268400 20749 log.cc:824] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:39.270253 20752 tablet_bootstrap.cc:492] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64: Bootstrap starting.
I20260929 15:51:39.271799 20749 tablet_bootstrap.cc:492] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904: No bootstrap required, opened a new log
I20260929 15:51:39.272096 20749 ts_tablet_manager.cc:1397] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260929 15:51:39.273005 20748 tablet_bootstrap.cc:492] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b: No bootstrap required, opened a new log
I20260929 15:51:39.273267 20748 ts_tablet_manager.cc:1397] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260929 15:51:39.273969 20752 tablet_bootstrap.cc:654] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:39.275137 20752 log.cc:824] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:39.278954 20752 tablet_bootstrap.cc:492] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64: No bootstrap required, opened a new log
I20260929 15:51:39.279219 20752 ts_tablet_manager.cc:1397] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260929 15:51:39.281045 20749 raft_consensus.cc:348] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef8f0d6d48d411f9fca75984345e72b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33411 } } peers { permanent_uuid: "c98d8a6dccfe4f8b8e62f1b922610a64" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34967 } } peers { permanent_uuid: "76eef21033994522bc51a6de63b57904" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46581 } }
I20260929 15:51:39.281397 20749 raft_consensus.cc:374] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:39.281523 20749 raft_consensus.cc:729] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76eef21033994522bc51a6de63b57904, State: Initialized, Role: FOLLOWER
I20260929 15:51:39.282120 20749 consensus_queue.cc:260] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef8f0d6d48d411f9fca75984345e72b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33411 } } peers { permanent_uuid: "c98d8a6dccfe4f8b8e62f1b922610a64" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34967 } } peers { permanent_uuid: "76eef21033994522bc51a6de63b57904" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46581 } }
I20260929 15:51:39.282485 20748 raft_consensus.cc:348] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef8f0d6d48d411f9fca75984345e72b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33411 } } peers { permanent_uuid: "c98d8a6dccfe4f8b8e62f1b922610a64" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34967 } } peers { permanent_uuid: "76eef21033994522bc51a6de63b57904" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46581 } }
I20260929 15:51:39.282886 20748 raft_consensus.cc:374] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:39.282995 20748 raft_consensus.cc:729] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef8f0d6d48d411f9fca75984345e72b, State: Initialized, Role: FOLLOWER
I20260929 15:51:39.283375 20729 heartbeater.cc:503] Master 127.15.196.254:46575 was elected leader, sending a full tablet report...
I20260929 15:51:39.283659 20748 consensus_queue.cc:260] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef8f0d6d48d411f9fca75984345e72b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33411 } } peers { permanent_uuid: "c98d8a6dccfe4f8b8e62f1b922610a64" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34967 } } peers { permanent_uuid: "76eef21033994522bc51a6de63b57904" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46581 } }
I20260929 15:51:39.283890 20749 ts_tablet_manager.cc:1428] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260929 15:51:39.284916 20596 heartbeater.cc:503] Master 127.15.196.254:46575 was elected leader, sending a full tablet report...
I20260929 15:51:39.285502 20748 ts_tablet_manager.cc:1428] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260929 15:51:39.289405 20752 raft_consensus.cc:348] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef8f0d6d48d411f9fca75984345e72b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33411 } } peers { permanent_uuid: "c98d8a6dccfe4f8b8e62f1b922610a64" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34967 } } peers { permanent_uuid: "76eef21033994522bc51a6de63b57904" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46581 } }
I20260929 15:51:39.289840 20752 raft_consensus.cc:374] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:39.289949 20752 raft_consensus.cc:729] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c98d8a6dccfe4f8b8e62f1b922610a64, State: Initialized, Role: FOLLOWER
I20260929 15:51:39.290416 20752 consensus_queue.cc:260] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef8f0d6d48d411f9fca75984345e72b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33411 } } peers { permanent_uuid: "c98d8a6dccfe4f8b8e62f1b922610a64" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34967 } } peers { permanent_uuid: "76eef21033994522bc51a6de63b57904" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46581 } }
I20260929 15:51:39.291903 20752 ts_tablet_manager.cc:1428] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
W20260929 15:51:39.324445 20464 tablet.cc:2369] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:51:39.368158 20755 raft_consensus.cc:482] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:51:39.368455 20755 raft_consensus.cc:504] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef8f0d6d48d411f9fca75984345e72b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33411 } } peers { permanent_uuid: "c98d8a6dccfe4f8b8e62f1b922610a64" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34967 } } peers { permanent_uuid: "76eef21033994522bc51a6de63b57904" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46581 } }
I20260929 15:51:39.370101 20755 leader_election.cc:288] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c98d8a6dccfe4f8b8e62f1b922610a64 (127.15.196.193:34967), 76eef21033994522bc51a6de63b57904 (127.15.196.195:46581)
W20260929 15:51:39.373533 20597 tablet.cc:2369] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:51:39.377005 20746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.370499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:56839 (local address 127.15.196.193:34967)
0929 15:51:39.370706 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:39.370712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:39.371257 (+ 545us) server_negotiation.cc:407] Connection header received
0929 15:51:39.371556 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:39.371563 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:39.371645 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:39.371729 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:39.372331 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.373155 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.375185 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.376041 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.376158 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.376210 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:39.376611 (+ 401us) server_negotiation.cc:299] Negotiation successful
0929 15:51:39.376740 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 15:51:39.377183 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.370297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34967 (local address 127.15.196.194:56839)
0929 15:51:39.371125 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:39.371147 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:39.371365 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:39.371770 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:39.371774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:39.371800 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:39.372181 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:39.372190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.373309 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.373316 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:39.375030 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:39.375042 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.376249 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.376256 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:39.376433 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.376472 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 15:51:39.376577 (+ 105us) client_negotiation.cc:240] Negotiation successful
0929 15:51:39.376671 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":11,"thread_start_us":462,"threads_started":1}
I20260929 15:51:39.377223 20745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.370715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:45751 (local address 127.15.196.195:46581)
0929 15:51:39.370881 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:39.370887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:39.371536 (+ 649us) server_negotiation.cc:407] Connection header received
0929 15:51:39.371677 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:39.371681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:39.371774 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:39.371868 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:39.372510 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.373224 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.375457 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.376287 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.376447 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.376516 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:39.376841 (+ 325us) server_negotiation.cc:299] Negotiation successful
0929 15:51:39.376968 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 15:51:39.377543 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.370547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:46581 (local address 127.15.196.194:45751)
0929 15:51:39.371406 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:39.371419 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:39.371556 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:39.371904 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:39.371907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:39.371949 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:39.372343 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:39.372355 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.373371 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.373376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:39.375291 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:39.375323 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.376483 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.376488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:39.376673 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.376705 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 15:51:39.376801 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 15:51:39.376924 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":59,"thread_start_us":540,"threads_started":1}
I20260929 15:51:39.378338 20418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b211359d4374d7c9ee0c244683e419e" candidate_uuid: "8ef8f0d6d48d411f9fca75984345e72b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c98d8a6dccfe4f8b8e62f1b922610a64" is_pre_election: true
I20260929 15:51:39.378571 20684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b211359d4374d7c9ee0c244683e419e" candidate_uuid: "8ef8f0d6d48d411f9fca75984345e72b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76eef21033994522bc51a6de63b57904" is_pre_election: true
I20260929 15:51:39.378949 20418 raft_consensus.cc:2454] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef8f0d6d48d411f9fca75984345e72b in term 0.
I20260929 15:51:39.379045 20684 raft_consensus.cc:2454] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef8f0d6d48d411f9fca75984345e72b in term 0.
I20260929 15:51:39.379727 20487 leader_election.cc:302] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 76eef21033994522bc51a6de63b57904, 8ef8f0d6d48d411f9fca75984345e72b, c98d8a6dccfe4f8b8e62f1b922610a64; no voters:
I20260929 15:51:39.380172 20755 raft_consensus.cc:2781] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:51:39.380314 20755 raft_consensus.cc:482] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:51:39.380414 20755 raft_consensus.cc:3037] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:39.382515 20755 raft_consensus.cc:504] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef8f0d6d48d411f9fca75984345e72b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33411 } } peers { permanent_uuid: "c98d8a6dccfe4f8b8e62f1b922610a64" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34967 } } peers { permanent_uuid: "76eef21033994522bc51a6de63b57904" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46581 } }
I20260929 15:51:39.383133 20755 leader_election.cc:288] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [CANDIDATE]: Term 1 election: Requested vote from peers c98d8a6dccfe4f8b8e62f1b922610a64 (127.15.196.193:34967), 76eef21033994522bc51a6de63b57904 (127.15.196.195:46581)
I20260929 15:51:39.383535 20418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b211359d4374d7c9ee0c244683e419e" candidate_uuid: "8ef8f0d6d48d411f9fca75984345e72b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c98d8a6dccfe4f8b8e62f1b922610a64"
I20260929 15:51:39.383747 20418 raft_consensus.cc:3037] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:39.383884 20684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b211359d4374d7c9ee0c244683e419e" candidate_uuid: "8ef8f0d6d48d411f9fca75984345e72b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76eef21033994522bc51a6de63b57904"
I20260929 15:51:39.384101 20684 raft_consensus.cc:3037] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:39.387110 20418 raft_consensus.cc:2454] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef8f0d6d48d411f9fca75984345e72b in term 1.
I20260929 15:51:39.387220 20684 raft_consensus.cc:2454] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef8f0d6d48d411f9fca75984345e72b in term 1.
I20260929 15:51:39.387665 20486 leader_election.cc:302] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76eef21033994522bc51a6de63b57904, 8ef8f0d6d48d411f9fca75984345e72b; no voters:
I20260929 15:51:39.388018 20755 raft_consensus.cc:2781] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:39.388441 20755 raft_consensus.cc:686] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [term 1 LEADER]: Becoming Leader. State: Replica: 8ef8f0d6d48d411f9fca75984345e72b, State: Running, Role: LEADER
I20260929 15:51:39.388955 20755 consensus_queue.cc:237] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef8f0d6d48d411f9fca75984345e72b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33411 } } peers { permanent_uuid: "c98d8a6dccfe4f8b8e62f1b922610a64" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34967 } } peers { permanent_uuid: "76eef21033994522bc51a6de63b57904" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46581 } }
I20260929 15:51:39.393847 20275 catalog_manager.cc:5495] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef8f0d6d48d411f9fca75984345e72b (127.15.196.194). New cstate: current_term: 1 leader_uuid: "8ef8f0d6d48d411f9fca75984345e72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ef8f0d6d48d411f9fca75984345e72b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c98d8a6dccfe4f8b8e62f1b922610a64" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76eef21033994522bc51a6de63b57904" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46581 } health_report { overall_health: UNKNOWN } } }
W20260929 15:51:39.397449 20730 tablet.cc:2369] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:51:39.439211 20743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.434350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48322 (local address 127.15.196.194:33411)
0929 15:51:39.434529 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:39.434534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:39.434551 (+ 17us) server_negotiation.cc:407] Connection header received
0929 15:51:39.434701 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:39.434704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:39.434789 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:39.434927 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:51:39.435221 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.435992 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.437269 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.437656 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.437748 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.438155 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:51:39.438662 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:51:39.438769 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:39.438838 (+ 69us) server_negotiation.cc:299] Negotiation successful
0929 15:51:39.438901 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 15:51:39.447476 20418 raft_consensus.cc:1261] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64 [term 1 FOLLOWER]: Refusing update from remote peer 8ef8f0d6d48d411f9fca75984345e72b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:39.447548 20684 raft_consensus.cc:1261] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904 [term 1 FOLLOWER]: Refusing update from remote peer 8ef8f0d6d48d411f9fca75984345e72b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:39.448500 20759 consensus_queue.cc:1035] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c98d8a6dccfe4f8b8e62f1b922610a64" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 34967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:39.448869 20755 consensus_queue.cc:1035] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [LEADER]: Connected to new peer: Peer: permanent_uuid: "76eef21033994522bc51a6de63b57904" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:39.460965 20762 mvcc.cc:205] Tried to move back new op lower bound from 7334695319326511104 to 7334695319101394944. Current Snapshot: MvccSnapshot[applied={T|T < 7334695319326511104}]
I20260929 15:51:39.478255 20275 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:39.478538 20275 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:39.482116 20275 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13abf444ff7c45e5a4454c459083ebff: Sending DeleteTablet for 3 replicas of tablet 1b211359d4374d7c9ee0c244683e419e
I20260929 15:51:39.483361 20531 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b211359d4374d7c9ee0c244683e419e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:39 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:56738
I20260929 15:51:39.483378 20398 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b211359d4374d7c9ee0c244683e419e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:39 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:34992
I20260929 15:51:39.483356 20664 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b211359d4374d7c9ee0c244683e419e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:39 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:46342
I20260929 15:51:39.485755 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 20336
I20260929 15:51:39.502228 20772 tablet_replica.cc:330] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b: stopping tablet replica
I20260929 15:51:39.502900 20773 tablet_replica.cc:330] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64: stopping tablet replica
I20260929 15:51:39.503834 20773 raft_consensus.cc:2229] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:39.504390 20773 raft_consensus.cc:2258] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:39.504905 20772 raft_consensus.cc:2229] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:51:39.505581 20772 raft_consensus.cc:2258] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:39.510121 20772 ts_tablet_manager.cc:1905] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 29 15:51:39 dist-test-slave-c51v krb5kdc[20231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:51:39.515825 20773 ts_tablet_manager.cc:1905] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:39.517328 20771 tablet_replica.cc:330] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904: stopping tablet replica
I20260929 15:51:39.517876 20771 raft_consensus.cc:2229] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:39.518445 20771 raft_consensus.cc:2258] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:39.521252 20771 ts_tablet_manager.cc:1905] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:39.522624 20772 ts_tablet_manager.cc:1918] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:51:39.522927 20772 log.cc:1196] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/1b211359d4374d7c9ee0c244683e419e
I20260929 15:51:39.523861 20772 ts_tablet_manager.cc:1939] T 1b211359d4374d7c9ee0c244683e419e P 8ef8f0d6d48d411f9fca75984345e72b: Deleting consensus metadata
I20260929 15:51:39.524953 20746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.487265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55698 (local address 127.15.196.193:34967)
0929 15:51:39.487477 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:39.487486 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:39.492227 (+ 4741us) server_negotiation.cc:407] Connection header received
0929 15:51:39.493286 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:39.493295 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:39.493527 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:39.493705 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:39.499051 (+ 5346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.500865 (+ 1814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.505063 (+ 4198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.506228 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.506405 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.515171 (+ 8766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:39.515202 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:39.515231 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:39.515361 (+ 130us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:39.520180 (+ 4819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:39.521417 (+ 1237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:39.521431 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:39.521435 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:39.521660 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:39.522255 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:39.522264 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:39.522267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:39.522892 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:39.523338 (+ 446us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:39.524084 (+ 746us) server_negotiation.cc:299] Negotiation successful
0929 15:51:39.524438 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260929 15:51:39.525635 20773 ts_tablet_manager.cc:1918] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:51:39.525810 20773 log.cc:1196] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/1b211359d4374d7c9ee0c244683e419e
I20260929 15:51:39.526662 20458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:39.526648 20262 catalog_manager.cc:4841] TS 8ef8f0d6d48d411f9fca75984345e72b (127.15.196.194:33411): tablet 1b211359d4374d7c9ee0c244683e419e (table test-table [id=951ee3668fe94d51aba7961b573c261d]) successfully deleted
I20260929 15:51:39.526969 20773 ts_tablet_manager.cc:1939] T 1b211359d4374d7c9ee0c244683e419e P c98d8a6dccfe4f8b8e62f1b922610a64: Deleting consensus metadata
I20260929 15:51:39.529191 20771 ts_tablet_manager.cc:1918] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:51:39.529378 20771 log.cc:1196] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/1b211359d4374d7c9ee0c244683e419e
I20260929 15:51:39.529932 20771 ts_tablet_manager.cc:1939] T 1b211359d4374d7c9ee0c244683e419e P 76eef21033994522bc51a6de63b57904: Deleting consensus metadata
I20260929 15:51:39.531810 20261 catalog_manager.cc:4841] TS 76eef21033994522bc51a6de63b57904 (127.15.196.195:46581): tablet 1b211359d4374d7c9ee0c244683e419e (table test-table [id=951ee3668fe94d51aba7961b573c261d]) successfully deleted
I20260929 15:51:39.629148 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 20336
W20260929 15:51:39.652194 20262 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.196.193:34967 (error 108)
I20260929 15:51:39.653045 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 20469
I20260929 15:51:39.654837 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.654296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34967 (local address 127.0.0.1:55700)
0929 15:51:39.654508 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:39.654647 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:34967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260929 15:51:39.655140 20262 catalog_manager.cc:4584] TS c98d8a6dccfe4f8b8e62f1b922610a64 (127.15.196.193:34967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1b211359d4374d7c9ee0c244683e419e: Network error: Client connection negotiation failed: client connection to 127.15.196.193:34967: connect: Connection refused (error 111)
Sep 29 15:51:39 dist-test-slave-c51v krb5kdc[20231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:51:39.671118 20743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.653993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48336 (local address 127.15.196.194:33411)
0929 15:51:39.654214 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:39.654222 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:39.654243 (+ 21us) server_negotiation.cc:407] Connection header received
0929 15:51:39.654393 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:39.654398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:39.654476 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:39.654581 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:39.656155 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.657414 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.658672 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.659380 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.659511 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.664090 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:39.664111 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:39.664138 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:39.664212 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:39.668126 (+ 3914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:39.668918 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:39.668928 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:39.668932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:39.669107 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:39.669493 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:39.669499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:39.669503 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:39.669978 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:39.670311 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:39.670492 (+ 181us) server_negotiation.cc:299] Negotiation successful
0929 15:51:39.670708 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 15:51:39.671952 20591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:39.709105 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.708599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34967 (local address 127.0.0.1:55702)
0929 15:51:39.708829 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:39.708930 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:34967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260929 15:51:39.710163 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.709827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34967 (local address 127.0.0.1:55718)
0929 15:51:39.709967 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:39.710049 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:34967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 15:51:39.770730 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.770243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34967 (local address 127.0.0.1:55720)
0929 15:51:39.770478 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:39.770578 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:34967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260929 15:51:39.771804 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.771483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34967 (local address 127.0.0.1:55732)
0929 15:51:39.771625 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:39.771692 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:34967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 15:51:39.788419 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 20469
I20260929 15:51:39.808672 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 20602
Sep 29 15:51:39 dist-test-slave-c51v krb5kdc[20231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:51:39.823495 20745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.809504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35446 (local address 127.15.196.195:46581)
0929 15:51:39.809682 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:39.809687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:39.809706 (+ 19us) server_negotiation.cc:407] Connection header received
0929 15:51:39.809814 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:39.809817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:39.809878 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:39.809963 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:39.811179 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.811970 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.813213 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:39.813585 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:39.813677 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:39.817893 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:39.817908 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:39.817925 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:39.817964 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:39.820727 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:39.821395 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:39.821403 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:39.821405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:39.821539 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:39.821921 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:39.821927 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:39.821930 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:39.822299 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:39.822592 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:39.822894 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 15:51:39.823127 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 15:51:39.824170 20724 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:39.852742 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.852264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34967 (local address 127.0.0.1:55734)
0929 15:51:39.852470 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:39.852559 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:34967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260929 15:51:39.853817 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:39.853433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34967 (local address 127.0.0.1:55750)
0929 15:51:39.853640 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:39.853707 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:34967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260929 15:51:39.914284 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 20602
I20260929 15:51:39.930573 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 20244
I20260929 15:51:39.931746 20305 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:40.041260 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:40.040407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:34967 (local address 127.0.0.1:55754)
0929 15:51:40.040731 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:40.040860 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:34967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260929 15:51:40.041451 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 20244
2026-09-29T15:51:40Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4331 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-29T15:51:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:51:40Z Disabled control of system clock
I20260929 15:51:40.108402 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:36899
--webserver_interface=127.15.196.254
--webserver_port=0
--builtin_ntp_servers=127.15.196.212:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:36899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 15:51:40.439205 20783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:40.439563 20783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:40.439649 20783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:40.447682 20783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:40.447858 20783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:40.447913 20783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:40.447980 20783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:40.448026 20783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:51:40.457556 20783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:36899
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:36899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:40.458850 20783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:40.460706 20783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:40.472956 20788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:40.473081 20791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:40.472996 20789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:40.473773 20783 server_base.cc:1034] running on GCE node
I20260929 15:51:40.474609 20783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:40.476181 20783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:40.477429 20783 hybrid_clock.cc:648] HybridClock initialized: now 1790697100477382 us; error 59 us; skew 500 ppm
I20260929 15:51:40.477811 20783 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:51:40.480157 20783 webserver.cc:460] Webserver started at http://127.15.196.254:35131/ using document root <none> and password file <none>
I20260929 15:51:40.480880 20783 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:40.481024 20783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:40.481366 20783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:40.484391 20783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "2773986f6dc14d7283fe38a3d98771db"
format_stamp: "Formatted at 2026-09-29 15:51:40 on dist-test-slave-c51v"
server_key: "22f2338ef387de371555ccd8487642b1"
server_key_iv: "f0de50b84553ce6dcc440d93ac691fe4"
server_key_version: "encryptionkey@0"
I20260929 15:51:40.485205 20783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "2773986f6dc14d7283fe38a3d98771db"
format_stamp: "Formatted at 2026-09-29 15:51:40 on dist-test-slave-c51v"
server_key: "22f2338ef387de371555ccd8487642b1"
server_key_iv: "f0de50b84553ce6dcc440d93ac691fe4"
server_key_version: "encryptionkey@0"
I20260929 15:51:40.490473 20783 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260929 15:51:40.494020 20797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:40.495441 20783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:40.495628 20783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "2773986f6dc14d7283fe38a3d98771db"
format_stamp: "Formatted at 2026-09-29 15:51:40 on dist-test-slave-c51v"
server_key: "22f2338ef387de371555ccd8487642b1"
server_key_iv: "f0de50b84553ce6dcc440d93ac691fe4"
server_key_version: "encryptionkey@0"
I20260929 15:51:40.495853 20783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:40.501508 20783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:40.502296 20783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:40.502601 20783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:40.521556 20783 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:36899
I20260929 15:51:40.521610 20848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:36899 every 8 connection(s)
I20260929 15:51:40.522950 20783 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:51:40.526666 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 20783
I20260929 15:51:40.526860 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:51:40.526993 20849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:40.527745 16147 external_mini_cluster.cc:1428] Setting key 08d819a4d9adf41d3f7fe6f2625c689b
I20260929 15:51:40.537178 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:40.529978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43926 (local address 127.15.196.254:36899)
0929 15:51:40.530996 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:40.531007 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:40.531055 (+ 48us) server_negotiation.cc:407] Connection header received
0929 15:51:40.533265 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:40.533289 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:40.533752 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:40.534203 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:40.535364 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:40.535371 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:40.535385 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:40.535401 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:40.535464 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:40.535486 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:40.535719 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:40.535855 (+ 136us) server_negotiation.cc:299] Negotiation successful
0929 15:51:40.536163 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":325,"threads_started":1}
I20260929 15:51:40.539996 20849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db: Bootstrap starting.
I20260929 15:51:40.543022 20849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:40.544104 20849 log.cc:824] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:40.547351 20849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db: No bootstrap required, opened a new log
I20260929 15:51:40.555474 20849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2773986f6dc14d7283fe38a3d98771db" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36899 } }
I20260929 15:51:40.556046 20849 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:40.556171 20849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2773986f6dc14d7283fe38a3d98771db, State: Initialized, Role: FOLLOWER
I20260929 15:51:40.556866 20849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2773986f6dc14d7283fe38a3d98771db" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36899 } }
I20260929 15:51:40.557080 20849 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:51:40.557178 20849 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:51:40.557302 20849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:40.559474 20849 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2773986f6dc14d7283fe38a3d98771db" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36899 } }
I20260929 15:51:40.559957 20849 leader_election.cc:302] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2773986f6dc14d7283fe38a3d98771db; no voters:
I20260929 15:51:40.560513 20849 leader_election.cc:288] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:51:40.560675 20854 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:40.561502 20854 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [term 1 LEADER]: Becoming Leader. State: Replica: 2773986f6dc14d7283fe38a3d98771db, State: Running, Role: LEADER
I20260929 15:51:40.562083 20854 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2773986f6dc14d7283fe38a3d98771db" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36899 } }
I20260929 15:51:40.562832 20849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:40.565250 20856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2773986f6dc14d7283fe38a3d98771db. Latest consensus state: current_term: 1 leader_uuid: "2773986f6dc14d7283fe38a3d98771db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2773986f6dc14d7283fe38a3d98771db" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36899 } } }
I20260929 15:51:40.565708 20856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:40.565250 20855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2773986f6dc14d7283fe38a3d98771db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2773986f6dc14d7283fe38a3d98771db" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36899 } } }
I20260929 15:51:40.565863 20855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:40.566835 20859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:40.572695 20859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:40.581928 20859 catalog_manager.cc:1347] Generated new cluster ID: 4c19db35c13c488bb00be6c8774e3cfe
I20260929 15:51:40.582044 20859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:40.601213 20859 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:51:40.601716 20859 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:40.622707 20859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db: Generated new TSK 0
I20260929 15:51:40.623829 20859 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:51:40.636564 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:36899
--builtin_ntp_servers=127.15.196.212:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 15:51:40.985742 20873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:40.986025 20873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:40.986176 20873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:40.993438 20873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:40.993538 20873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:40.993731 20873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:51:41.003103 20873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:36899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:41.004518 20873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:41.006275 20873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:41.019822 20878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:41.019769 20881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:41.019821 20879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:41.024987 20873 server_base.cc:1034] running on GCE node
I20260929 15:51:41.025655 20873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:41.026700 20873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:41.027936 20873 hybrid_clock.cc:648] HybridClock initialized: now 1790697101027877 us; error 52 us; skew 500 ppm
I20260929 15:51:41.028281 20873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:51:41.030602 20873 webserver.cc:460] Webserver started at http://127.15.196.193:40025/ using document root <none> and password file <none>
I20260929 15:51:41.031404 20873 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:41.031548 20873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:41.031896 20873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:41.034819 20873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "137185f289b5460e93cee404d3aea658"
format_stamp: "Formatted at 2026-09-29 15:51:41 on dist-test-slave-c51v"
server_key: "4dfc7c6a29ea8b6ca0483eb6be388c6d"
server_key_iv: "635b6a7d3575dee129310389cedad886"
server_key_version: "encryptionkey@0"
I20260929 15:51:41.035703 20873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "137185f289b5460e93cee404d3aea658"
format_stamp: "Formatted at 2026-09-29 15:51:41 on dist-test-slave-c51v"
server_key: "4dfc7c6a29ea8b6ca0483eb6be388c6d"
server_key_iv: "635b6a7d3575dee129310389cedad886"
server_key_version: "encryptionkey@0"
I20260929 15:51:41.041277 20873 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260929 15:51:41.044781 20887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:41.046196 20873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 15:51:41.046391 20873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "137185f289b5460e93cee404d3aea658"
format_stamp: "Formatted at 2026-09-29 15:51:41 on dist-test-slave-c51v"
server_key: "4dfc7c6a29ea8b6ca0483eb6be388c6d"
server_key_iv: "635b6a7d3575dee129310389cedad886"
server_key_version: "encryptionkey@0"
I20260929 15:51:41.046640 20873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:41.052568 20873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:41.053413 20873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:41.053694 20873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:41.054605 20873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:41.056121 20873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:41.056248 20873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:41.056373 20873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:41.056463 20873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:41.090133 20873 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:39475
I20260929 15:51:41.090183 20999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:39475 every 8 connection(s)
I20260929 15:51:41.091607 20873 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:51:41.099359 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 20873
I20260929 15:51:41.099582 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:51:41.100181 21002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:41.094237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:52865 (local address 127.15.196.254:36899)
0929 15:51:41.094721 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:41.094725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:41.095935 (+ 1210us) server_negotiation.cc:407] Connection header received
0929 15:51:41.097662 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:41.097668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:41.097752 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:41.097873 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:41.099399 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:41.099403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:41.099406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:41.099432 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:41.099512 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:41.099528 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:41.099727 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:41.099906 (+ 179us) server_negotiation.cc:299] Negotiation successful
0929 15:51:41.099961 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":238,"threads_started":1}
I20260929 15:51:41.100414 16147 external_mini_cluster.cc:1428] Setting key 67d6564003c0a1468a62149c9412a647
I20260929 15:51:41.101387 21003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:41.094877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36899 (local address 127.15.196.193:52865)
0929 15:51:41.095777 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:41.095814 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:41.097373 (+ 1559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:41.098128 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:41.098136 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:41.098228 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:41.098968 (+ 740us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:51:41.098973 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:41.099050 (+ 77us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:51:41.099168 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:41.099674 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:41.099685 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:41.099694 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 15:51:41.100000 (+ 306us) client_negotiation.cc:240] Negotiation successful
0929 15:51:41.100291 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":252,"threads_started":1}
I20260929 15:51:41.102895 21000 heartbeater.cc:347] Connected to a master server at 127.15.196.254:36899
I20260929 15:51:41.103193 21000 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:41.103344 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:36899
--builtin_ntp_servers=127.15.196.212:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260929 15:51:41.103890 21000 heartbeater.cc:511] Master 127.15.196.254:36899 requested a full tablet report, sending...
I20260929 15:51:41.105840 20813 ts_manager.cc:194] Registered new tserver with Master: 137185f289b5460e93cee404d3aea658 (127.15.196.193:39475)
W20260929 15:51:41.429208 21004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:41.429498 21004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:41.429589 21004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:41.436880 21004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:41.436985 21004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:41.437098 21004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:51:41.446241 21004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:36899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:41.447542 21004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:41.449280 21004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:41.462462 21012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:41.462416 21010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:41.462838 21009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:41.463115 21004 server_base.cc:1034] running on GCE node
I20260929 15:51:41.463796 21004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:41.464730 21004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:41.465960 21004 hybrid_clock.cc:648] HybridClock initialized: now 1790697101465909 us; error 44 us; skew 500 ppm
I20260929 15:51:41.466279 21004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:51:41.473623 21004 webserver.cc:460] Webserver started at http://127.15.196.194:44115/ using document root <none> and password file <none>
I20260929 15:51:41.474359 21004 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:41.474509 21004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:41.474848 21004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:41.477810 21004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "4c6f62c87e1346978aae323f34941727"
format_stamp: "Formatted at 2026-09-29 15:51:41 on dist-test-slave-c51v"
server_key: "281f33a18c6b3a4c8c3914b966f7495c"
server_key_iv: "f4a3fb2daaba60d3196ad19f54c91c08"
server_key_version: "encryptionkey@0"
I20260929 15:51:41.478606 21004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "4c6f62c87e1346978aae323f34941727"
format_stamp: "Formatted at 2026-09-29 15:51:41 on dist-test-slave-c51v"
server_key: "281f33a18c6b3a4c8c3914b966f7495c"
server_key_iv: "f4a3fb2daaba60d3196ad19f54c91c08"
server_key_version: "encryptionkey@0"
I20260929 15:51:41.483839 21004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 15:51:41.487146 21018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:41.488492 21004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:41.488669 21004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "4c6f62c87e1346978aae323f34941727"
format_stamp: "Formatted at 2026-09-29 15:51:41 on dist-test-slave-c51v"
server_key: "281f33a18c6b3a4c8c3914b966f7495c"
server_key_iv: "f4a3fb2daaba60d3196ad19f54c91c08"
server_key_version: "encryptionkey@0"
I20260929 15:51:41.488884 21004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:41.494556 21004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:41.495410 21004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:41.495674 21004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:41.496542 21004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:41.497928 21004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:41.498054 21004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:41.498176 21004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:41.498260 21004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:41.531518 21004 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:38567
I20260929 15:51:41.531603 21130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:38567 every 8 connection(s)
I20260929 15:51:41.532879 21004 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:51:41.541975 21002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:41.535682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:55043 (local address 127.15.196.254:36899)
0929 15:51:41.535882 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:41.535888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:41.537381 (+ 1493us) server_negotiation.cc:407] Connection header received
0929 15:51:41.539779 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:41.539783 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:41.539839 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:41.539930 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:41.541359 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:41.541363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:41.541366 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:41.541383 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:41.541414 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:41.541437 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:41.541543 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:41.541720 (+ 177us) server_negotiation.cc:299] Negotiation successful
0929 15:51:41.541770 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 15:51:41.542994 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 21004
I20260929 15:51:41.543032 21133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:41.536202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36899 (local address 127.15.196.194:55043)
0929 15:51:41.537208 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:41.537255 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:41.539488 (+ 2233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:41.540190 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:41.540201 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:41.540327 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:41.541048 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:51:41.541051 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:41.541090 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:51:41.541179 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:41.541539 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:41.541546 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:41.541552 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 15:51:41.541773 (+ 221us) client_negotiation.cc:240] Negotiation successful
0929 15:51:41.541997 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":267,"threads_started":1}
I20260929 15:51:41.543159 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:51:41.543735 16147 external_mini_cluster.cc:1428] Setting key 0235198ba6411066a6133e934cdd6376
I20260929 15:51:41.544706 21131 heartbeater.cc:347] Connected to a master server at 127.15.196.254:36899
I20260929 15:51:41.545024 21131 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:41.545759 21131 heartbeater.cc:511] Master 127.15.196.254:36899 requested a full tablet report, sending...
I20260929 15:51:41.546931 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:36899
--builtin_ntp_servers=127.15.196.212:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260929 15:51:41.547214 20813 ts_manager.cc:194] Registered new tserver with Master: 4c6f62c87e1346978aae323f34941727 (127.15.196.194:38567)
W20260929 15:51:41.883872 21134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:41.884223 21134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:41.884402 21134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:41.892272 21134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:41.892371 21134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:41.892557 21134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:51:41.902158 21134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:36899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:41.903548 21134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:41.905346 21134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:41.918678 21142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:41.918841 21139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:41.918866 21140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:41.919801 21134 server_base.cc:1034] running on GCE node
I20260929 15:51:41.920387 21134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:41.921314 21134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:41.922537 21134 hybrid_clock.cc:648] HybridClock initialized: now 1790697101922487 us; error 65 us; skew 500 ppm
I20260929 15:51:41.922848 21134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:51:41.925169 21134 webserver.cc:460] Webserver started at http://127.15.196.195:38297/ using document root <none> and password file <none>
I20260929 15:51:41.925855 21134 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:41.925992 21134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:41.926327 21134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:41.929204 21134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "0e7943dfd7ec47b9a92f46956da428f0"
format_stamp: "Formatted at 2026-09-29 15:51:41 on dist-test-slave-c51v"
server_key: "53efe579f2e70df498fe3a2b00c0c6c7"
server_key_iv: "62f0ad8e3f04f2040b66f8db59bab10b"
server_key_version: "encryptionkey@0"
I20260929 15:51:41.929989 21134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "0e7943dfd7ec47b9a92f46956da428f0"
format_stamp: "Formatted at 2026-09-29 15:51:41 on dist-test-slave-c51v"
server_key: "53efe579f2e70df498fe3a2b00c0c6c7"
server_key_iv: "62f0ad8e3f04f2040b66f8db59bab10b"
server_key_version: "encryptionkey@0"
I20260929 15:51:41.934989 21134 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260929 15:51:41.938335 21148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:41.939739 21134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:41.939931 21134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "0e7943dfd7ec47b9a92f46956da428f0"
format_stamp: "Formatted at 2026-09-29 15:51:41 on dist-test-slave-c51v"
server_key: "53efe579f2e70df498fe3a2b00c0c6c7"
server_key_iv: "62f0ad8e3f04f2040b66f8db59bab10b"
server_key_version: "encryptionkey@0"
I20260929 15:51:41.940165 21134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:41.945964 21134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:41.946794 21134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:41.947080 21134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:41.948127 21134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:41.949590 21134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:41.949712 21134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:41.949846 21134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:41.949932 21134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:41.985571 21134 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:38209
I20260929 15:51:41.985623 21260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:38209 every 8 connection(s)
I20260929 15:51:41.987241 21134 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:51:41.996460 21002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:41.990094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:49107 (local address 127.15.196.254:36899)
0929 15:51:41.990278 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:41.990283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:41.991831 (+ 1548us) server_negotiation.cc:407] Connection header received
0929 15:51:41.993866 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:41.993872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:41.993961 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:41.994087 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:41.995551 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:41.995557 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:41.995562 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:41.995592 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:41.995626 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:41.995639 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:41.995771 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:41.995984 (+ 213us) server_negotiation.cc:299] Negotiation successful
0929 15:51:41.996059 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 15:51:41.996692 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 21134
I20260929 15:51:41.996881 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:51:41.997321 16147 external_mini_cluster.cc:1428] Setting key 79c5cf53d8cd27deb2d410012aeaeced
I20260929 15:51:41.997596 21263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:41.990643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36899 (local address 127.15.196.195:49107)
0929 15:51:41.991671 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:41.991708 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:41.993572 (+ 1864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:41.994328 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:41.994337 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:41.994439 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:41.995202 (+ 763us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:51:41.995205 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:41.995246 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:51:41.995361 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:41.995740 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:41.995751 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:41.995759 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 15:51:41.996073 (+ 314us) client_negotiation.cc:240] Negotiation successful
0929 15:51:41.996347 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":300,"threads_started":1}
I20260929 15:51:41.999018 21261 heartbeater.cc:347] Connected to a master server at 127.15.196.254:36899
I20260929 15:51:41.999372 21261 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:42.000051 21261 heartbeater.cc:511] Master 127.15.196.254:36899 requested a full tablet report, sending...
I20260929 15:51:42.001369 20814 ts_manager.cc:194] Registered new tserver with Master: 0e7943dfd7ec47b9a92f46956da428f0 (127.15.196.195:38209)
I20260929 15:51:42.012593 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:51:42.019495 21002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.017168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43938 (local address 127.15.196.254:36899)
0929 15:51:42.017328 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:42.017332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:42.017883 (+ 551us) server_negotiation.cc:407] Connection header received
0929 15:51:42.018014 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:42.018018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:42.018069 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:42.018147 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:42.018971 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:42.018975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:42.018978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:42.018994 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:42.019022 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:42.019034 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:42.019125 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:42.019181 (+ 56us) server_negotiation.cc:299] Negotiation successful
0929 15:51:42.019224 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 15:51:42.023118 20814 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:51:42.029831 20814 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:51:42.050720 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.041946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:38209 (local address 127.0.0.1:35368)
0929 15:51:42.043884 (+ 1938us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:42.043941 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:42.044103 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:42.045610 (+ 1507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:42.045621 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:42.045734 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:42.047989 (+ 2255us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:51:42.047996 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:42.048037 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:51:42.048152 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:42.048862 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:42.048875 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:42.048888 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 15:51:42.049100 (+ 212us) client_negotiation.cc:240] Negotiation successful
0929 15:51:42.049220 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1624,"threads_started":1}
I20260929 15:51:42.052124 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.042154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35368 (local address 127.15.196.195:38209)
0929 15:51:42.044501 (+ 2347us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:42.044511 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:42.044544 (+ 33us) server_negotiation.cc:407] Connection header received
0929 15:51:42.044616 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:42.044623 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:42.044864 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:42.045021 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:42.048408 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:42.048421 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:42.048442 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:42.048475 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:42.048547 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:42.048572 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:42.048802 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:42.051414 (+ 2612us) server_negotiation.cc:299] Negotiation successful
0929 15:51:42.051561 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2112,"thread_start_us":957,"threads_started":1}
I20260929 15:51:42.057691 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.042893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35938 (local address 127.15.196.193:39475)
0929 15:51:42.053559 (+ 10666us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:42.053572 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:42.053612 (+ 40us) server_negotiation.cc:407] Connection header received
0929 15:51:42.053698 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:42.053709 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:42.054032 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:42.054215 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:42.056776 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:42.056786 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:42.056801 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:42.056821 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:42.056884 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:42.056908 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:42.057062 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:42.057261 (+ 199us) server_negotiation.cc:299] Negotiation successful
0929 15:51:42.057363 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10455,"thread_start_us":13574,"threads_started":1}
I20260929 15:51:42.057727 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.042058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:39475 (local address 127.0.0.1:35938)
0929 15:51:42.050321 (+ 8263us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:42.050348 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:42.050485 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:42.055425 (+ 4940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:42.055433 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:42.055467 (+ 34us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:42.056471 (+ 1004us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:51:42.056478 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:42.056502 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:51:42.056566 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:42.057097 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:42.057102 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:42.057108 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 15:51:42.057264 (+ 156us) client_negotiation.cc:240] Negotiation successful
0929 15:51:42.057318 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8063,"thread_start_us":1407,"threads_started":1}
I20260929 15:51:42.057585 21196 tablet_service.cc:1467] Processing CreateTablet for tablet 4c639bc817fe4ab9be000ea7b8f3a4e3 (DEFAULT_TABLE table=test-table [id=e15318c7fb9849faaa03bf3dbc6dd11e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:42.059266 21196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c639bc817fe4ab9be000ea7b8f3a4e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:42.060362 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.044675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:38567 (local address 127.0.0.1:55118)
0929 15:51:42.052650 (+ 7975us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:42.052674 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:42.052774 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:42.058300 (+ 5526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:42.058304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:42.058328 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:42.058868 (+ 540us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:51:42.058872 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:42.058888 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:51:42.058929 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:42.059930 (+ 1001us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:42.059935 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:42.059940 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 15:51:42.060018 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 15:51:42.060068 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7833,"thread_start_us":10543,"threads_started":1}
I20260929 15:51:42.061213 21278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.044874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55118 (local address 127.15.196.194:38567)
0929 15:51:42.057457 (+ 12583us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:42.057465 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:42.057495 (+ 30us) server_negotiation.cc:407] Connection header received
0929 15:51:42.057582 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:42.057591 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:42.058137 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:42.058313 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:42.059675 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:42.059683 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:42.059696 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:42.059721 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:42.059778 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:42.059800 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:42.060707 (+ 907us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:42.060778 (+ 71us) server_negotiation.cc:299] Negotiation successful
0929 15:51:42.060883 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12084,"thread_start_us":11959,"threads_started":1}
I20260929 15:51:42.061666 20935 tablet_service.cc:1467] Processing CreateTablet for tablet 4c639bc817fe4ab9be000ea7b8f3a4e3 (DEFAULT_TABLE table=test-table [id=e15318c7fb9849faaa03bf3dbc6dd11e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:42.064054 20935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c639bc817fe4ab9be000ea7b8f3a4e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:42.064801 21066 tablet_service.cc:1467] Processing CreateTablet for tablet 4c639bc817fe4ab9be000ea7b8f3a4e3 (DEFAULT_TABLE table=test-table [id=e15318c7fb9849faaa03bf3dbc6dd11e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:42.066144 21066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c639bc817fe4ab9be000ea7b8f3a4e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:42.071698 21279 tablet_bootstrap.cc:492] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0: Bootstrap starting.
I20260929 15:51:42.074949 21279 tablet_bootstrap.cc:654] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:42.076234 21279 log.cc:824] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:42.079000 21279 tablet_bootstrap.cc:492] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0: No bootstrap required, opened a new log
I20260929 15:51:42.079141 21281 tablet_bootstrap.cc:492] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 137185f289b5460e93cee404d3aea658: Bootstrap starting.
I20260929 15:51:42.079794 21279 ts_tablet_manager.cc:1397] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260929 15:51:42.083032 21282 tablet_bootstrap.cc:492] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727: Bootstrap starting.
I20260929 15:51:42.084182 21281 tablet_bootstrap.cc:654] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 137185f289b5460e93cee404d3aea658: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:42.085444 21281 log.cc:824] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 137185f289b5460e93cee404d3aea658: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:42.086179 21282 tablet_bootstrap.cc:654] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:42.087369 21282 log.cc:824] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:42.088707 21281 tablet_bootstrap.cc:492] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 137185f289b5460e93cee404d3aea658: No bootstrap required, opened a new log
I20260929 15:51:42.088981 21281 ts_tablet_manager.cc:1397] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 137185f289b5460e93cee404d3aea658: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260929 15:51:42.090018 21282 tablet_bootstrap.cc:492] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727: No bootstrap required, opened a new log
I20260929 15:51:42.089792 21279 raft_consensus.cc:348] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7943dfd7ec47b9a92f46956da428f0" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38209 } } peers { permanent_uuid: "137185f289b5460e93cee404d3aea658" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 39475 } } peers { permanent_uuid: "4c6f62c87e1346978aae323f34941727" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38567 } }
I20260929 15:51:42.090320 21282 ts_tablet_manager.cc:1397] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.004s
I20260929 15:51:42.090369 21279 raft_consensus.cc:374] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:42.090564 21279 raft_consensus.cc:729] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7943dfd7ec47b9a92f46956da428f0, State: Initialized, Role: FOLLOWER
I20260929 15:51:42.091390 21279 consensus_queue.cc:260] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7943dfd7ec47b9a92f46956da428f0" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38209 } } peers { permanent_uuid: "137185f289b5460e93cee404d3aea658" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 39475 } } peers { permanent_uuid: "4c6f62c87e1346978aae323f34941727" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38567 } }
I20260929 15:51:42.098815 21261 heartbeater.cc:503] Master 127.15.196.254:36899 was elected leader, sending a full tablet report...
I20260929 15:51:42.099817 21279 ts_tablet_manager.cc:1428] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0: Time spent starting tablet: real 0.020s user 0.013s sys 0.006s
I20260929 15:51:42.100883 21282 raft_consensus.cc:348] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7943dfd7ec47b9a92f46956da428f0" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38209 } } peers { permanent_uuid: "137185f289b5460e93cee404d3aea658" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 39475 } } peers { permanent_uuid: "4c6f62c87e1346978aae323f34941727" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38567 } }
I20260929 15:51:42.101433 21282 raft_consensus.cc:374] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:42.101572 21282 raft_consensus.cc:729] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c6f62c87e1346978aae323f34941727, State: Initialized, Role: FOLLOWER
I20260929 15:51:42.103003 21282 consensus_queue.cc:260] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7943dfd7ec47b9a92f46956da428f0" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38209 } } peers { permanent_uuid: "137185f289b5460e93cee404d3aea658" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 39475 } } peers { permanent_uuid: "4c6f62c87e1346978aae323f34941727" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38567 } }
I20260929 15:51:42.103935 21281 raft_consensus.cc:348] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 137185f289b5460e93cee404d3aea658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7943dfd7ec47b9a92f46956da428f0" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38209 } } peers { permanent_uuid: "137185f289b5460e93cee404d3aea658" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 39475 } } peers { permanent_uuid: "4c6f62c87e1346978aae323f34941727" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38567 } }
I20260929 15:51:42.104393 21281 raft_consensus.cc:374] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 137185f289b5460e93cee404d3aea658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:42.104525 21281 raft_consensus.cc:729] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 137185f289b5460e93cee404d3aea658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 137185f289b5460e93cee404d3aea658, State: Initialized, Role: FOLLOWER
I20260929 15:51:42.105206 21281 consensus_queue.cc:260] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 137185f289b5460e93cee404d3aea658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7943dfd7ec47b9a92f46956da428f0" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38209 } } peers { permanent_uuid: "137185f289b5460e93cee404d3aea658" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 39475 } } peers { permanent_uuid: "4c6f62c87e1346978aae323f34941727" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38567 } }
I20260929 15:51:42.106606 21131 heartbeater.cc:503] Master 127.15.196.254:36899 was elected leader, sending a full tablet report...
I20260929 15:51:42.107240 21282 ts_tablet_manager.cc:1428] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727: Time spent starting tablet: real 0.017s user 0.010s sys 0.005s
I20260929 15:51:42.108187 21000 heartbeater.cc:503] Master 127.15.196.254:36899 was elected leader, sending a full tablet report...
I20260929 15:51:42.108745 21281 ts_tablet_manager.cc:1428] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 137185f289b5460e93cee404d3aea658: Time spent starting tablet: real 0.020s user 0.016s sys 0.002s
I20260929 15:51:42.142372 21286 raft_consensus.cc:482] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:51:42.142748 21286 raft_consensus.cc:504] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7943dfd7ec47b9a92f46956da428f0" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38209 } } peers { permanent_uuid: "137185f289b5460e93cee404d3aea658" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 39475 } } peers { permanent_uuid: "4c6f62c87e1346978aae323f34941727" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38567 } }
I20260929 15:51:42.144640 21286 leader_election.cc:288] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e7943dfd7ec47b9a92f46956da428f0 (127.15.196.195:38209), 137185f289b5460e93cee404d3aea658 (127.15.196.193:39475)
I20260929 15:51:42.148329 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.145099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:57609 (local address 127.15.196.193:39475)
0929 15:51:42.145307 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:42.145313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:42.146063 (+ 750us) server_negotiation.cc:407] Connection header received
0929 15:51:42.146280 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:42.146287 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:42.146350 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:42.146470 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:42.147687 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:42.147693 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:42.147697 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:42.147716 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:42.147754 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:42.147770 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:42.147891 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:42.148001 (+ 110us) server_negotiation.cc:299] Negotiation successful
0929 15:51:42.148062 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260929 15:51:42.148329 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.145203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:57877 (local address 127.15.196.195:38209)
0929 15:51:42.145366 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:42.145373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:42.146062 (+ 689us) server_negotiation.cc:407] Connection header received
0929 15:51:42.146280 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:42.146287 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:42.146350 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:42.146463 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:42.147704 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:42.147710 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:42.147714 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:42.147732 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:42.147763 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:42.147776 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:42.147894 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:42.148002 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 15:51:42.148062 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 15:51:42.148660 21288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.144844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:38209 (local address 127.15.196.194:57877)
0929 15:51:42.145909 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:42.145939 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:42.146114 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:42.146503 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:42.146508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:42.146533 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:42.147526 (+ 993us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:51:42.147528 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:42.147540 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:51:42.147575 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:42.147875 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:42.147880 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:42.147885 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 15:51:42.148053 (+ 168us) client_negotiation.cc:240] Negotiation successful
0929 15:51:42.148132 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"mutex_wait_us":21,"spinlock_wait_cycles":35712,"thread_start_us":683,"threads_started":1}
I20260929 15:51:42.148787 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.145081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:39475 (local address 127.15.196.194:57609)
0929 15:51:42.145945 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:42.145969 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:42.146077 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:42.146503 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:42.146508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:42.146533 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:42.147431 (+ 898us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:51:42.147436 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:42.147456 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:51:42.147515 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:42.147875 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:42.147880 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:42.147885 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 15:51:42.148060 (+ 175us) client_negotiation.cc:240] Negotiation successful
0929 15:51:42.148119 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"thread_start_us":466,"threads_started":1}
I20260929 15:51:42.149695 20955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c639bc817fe4ab9be000ea7b8f3a4e3" candidate_uuid: "4c6f62c87e1346978aae323f34941727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "137185f289b5460e93cee404d3aea658" is_pre_election: true
I20260929 15:51:42.149739 21216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c639bc817fe4ab9be000ea7b8f3a4e3" candidate_uuid: "4c6f62c87e1346978aae323f34941727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e7943dfd7ec47b9a92f46956da428f0" is_pre_election: true
I20260929 15:51:42.150233 20955 raft_consensus.cc:2454] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 137185f289b5460e93cee404d3aea658 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c6f62c87e1346978aae323f34941727 in term 0.
I20260929 15:51:42.150297 21216 raft_consensus.cc:2454] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c6f62c87e1346978aae323f34941727 in term 0.
I20260929 15:51:42.150920 21019 leader_election.cc:302] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e7943dfd7ec47b9a92f46956da428f0, 4c6f62c87e1346978aae323f34941727; no voters:
I20260929 15:51:42.151350 21286 raft_consensus.cc:2781] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:51:42.151520 21286 raft_consensus.cc:482] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:51:42.151633 21286 raft_consensus.cc:3037] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:42.153924 21286 raft_consensus.cc:504] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7943dfd7ec47b9a92f46956da428f0" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38209 } } peers { permanent_uuid: "137185f289b5460e93cee404d3aea658" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 39475 } } peers { permanent_uuid: "4c6f62c87e1346978aae323f34941727" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38567 } }
I20260929 15:51:42.154637 21286 leader_election.cc:288] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [CANDIDATE]: Term 1 election: Requested vote from peers 0e7943dfd7ec47b9a92f46956da428f0 (127.15.196.195:38209), 137185f289b5460e93cee404d3aea658 (127.15.196.193:39475)
I20260929 15:51:42.154979 21216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c639bc817fe4ab9be000ea7b8f3a4e3" candidate_uuid: "4c6f62c87e1346978aae323f34941727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e7943dfd7ec47b9a92f46956da428f0"
I20260929 15:51:42.155184 21216 raft_consensus.cc:3037] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:42.155345 20955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c639bc817fe4ab9be000ea7b8f3a4e3" candidate_uuid: "4c6f62c87e1346978aae323f34941727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "137185f289b5460e93cee404d3aea658"
I20260929 15:51:42.155561 20955 raft_consensus.cc:3037] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 137185f289b5460e93cee404d3aea658 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:42.157981 21216 raft_consensus.cc:2454] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c6f62c87e1346978aae323f34941727 in term 1.
I20260929 15:51:42.158331 20955 raft_consensus.cc:2454] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 137185f289b5460e93cee404d3aea658 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c6f62c87e1346978aae323f34941727 in term 1.
I20260929 15:51:42.158361 21019 leader_election.cc:302] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e7943dfd7ec47b9a92f46956da428f0, 4c6f62c87e1346978aae323f34941727; no voters:
I20260929 15:51:42.158628 21286 raft_consensus.cc:2781] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:42.159102 21286 raft_consensus.cc:686] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [term 1 LEADER]: Becoming Leader. State: Replica: 4c6f62c87e1346978aae323f34941727, State: Running, Role: LEADER
I20260929 15:51:42.159646 21286 consensus_queue.cc:237] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7943dfd7ec47b9a92f46956da428f0" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38209 } } peers { permanent_uuid: "137185f289b5460e93cee404d3aea658" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 39475 } } peers { permanent_uuid: "4c6f62c87e1346978aae323f34941727" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38567 } }
I20260929 15:51:42.164011 20813 catalog_manager.cc:5495] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c6f62c87e1346978aae323f34941727 (127.15.196.194). New cstate: current_term: 1 leader_uuid: "4c6f62c87e1346978aae323f34941727" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7943dfd7ec47b9a92f46956da428f0" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "137185f289b5460e93cee404d3aea658" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 39475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c6f62c87e1346978aae323f34941727" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38567 } health_report { overall_health: HEALTHY } } }
I20260929 15:51:42.183912 21278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.181958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55132 (local address 127.15.196.194:38567)
0929 15:51:42.182114 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:42.182118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:42.182211 (+ 93us) server_negotiation.cc:407] Connection header received
0929 15:51:42.182371 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:42.182374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:42.182453 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:42.182533 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:42.183450 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:42.183454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:42.183457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:42.183470 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:42.183497 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:42.183507 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:42.183593 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:42.183663 (+ 70us) server_negotiation.cc:299] Negotiation successful
0929 15:51:42.183707 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 15:51:42.192096 20955 raft_consensus.cc:1261] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 137185f289b5460e93cee404d3aea658 [term 1 FOLLOWER]: Refusing update from remote peer 4c6f62c87e1346978aae323f34941727: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:42.192096 21216 raft_consensus.cc:1261] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0 [term 1 FOLLOWER]: Refusing update from remote peer 4c6f62c87e1346978aae323f34941727: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:42.193163 21290 consensus_queue.cc:1035] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e7943dfd7ec47b9a92f46956da428f0" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:42.193549 21286 consensus_queue.cc:1035] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [LEADER]: Connected to new peer: Peer: permanent_uuid: "137185f289b5460e93cee404d3aea658" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 39475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:42.204594 21293 mvcc.cc:205] Tried to move back new op lower bound from 7334695330567892992 to 7334695330450100224. Current Snapshot: MvccSnapshot[applied={T|T < 7334695330567892992}]
I20260929 15:51:42.223124 20814 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:42.223443 20814 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:42.227041 20814 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2773986f6dc14d7283fe38a3d98771db: Sending DeleteTablet for 3 replicas of tablet 4c639bc817fe4ab9be000ea7b8f3a4e3
I20260929 15:51:42.228257 21196 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c639bc817fe4ab9be000ea7b8f3a4e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:42 UTC) from {username='slave'} at 127.0.0.1:35368
I20260929 15:51:42.228271 21066 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c639bc817fe4ab9be000ea7b8f3a4e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:42 UTC) from {username='slave'} at 127.0.0.1:55118
I20260929 15:51:42.228262 20935 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c639bc817fe4ab9be000ea7b8f3a4e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:42 UTC) from {username='slave'} at 127.0.0.1:35938
I20260929 15:51:42.233177 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 20873
I20260929 15:51:42.255494 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.234952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35952 (local address 127.15.196.193:39475)
0929 15:51:42.235136 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:42.235142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:42.252607 (+ 17465us) server_negotiation.cc:407] Connection header received
0929 15:51:42.253140 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:42.253146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:42.253273 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:42.253355 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:42.254588 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:42.254593 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:42.254598 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:42.254620 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:42.254658 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:42.254677 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:42.254779 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:42.255034 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 15:51:42.255100 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 15:51:42.256884 20995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:42.257547 21304 tablet_replica.cc:330] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0: stopping tablet replica
I20260929 15:51:42.258204 21304 raft_consensus.cc:2229] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:42.258848 21304 raft_consensus.cc:2258] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:42.261240 21304 ts_tablet_manager.cc:1905] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:42.261653 21302 tablet_replica.cc:330] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727: stopping tablet replica
I20260929 15:51:42.262069 21302 raft_consensus.cc:2229] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:51:42.262709 21302 raft_consensus.cc:2258] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:42.265410 21302 ts_tablet_manager.cc:1905] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:42.268625 21304 ts_tablet_manager.cc:1918] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:51:42.268850 21304 log.cc:1196] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/4c639bc817fe4ab9be000ea7b8f3a4e3
I20260929 15:51:42.269464 21304 ts_tablet_manager.cc:1939] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 0e7943dfd7ec47b9a92f46956da428f0: Deleting consensus metadata
I20260929 15:51:42.271473 20798 catalog_manager.cc:4841] TS 0e7943dfd7ec47b9a92f46956da428f0 (127.15.196.195:38209): tablet 4c639bc817fe4ab9be000ea7b8f3a4e3 (table test-table [id=e15318c7fb9849faaa03bf3dbc6dd11e]) successfully deleted
I20260929 15:51:42.278450 21302 ts_tablet_manager.cc:1918] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:51:42.278869 21302 log.cc:1196] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/4c639bc817fe4ab9be000ea7b8f3a4e3
I20260929 15:51:42.279583 21302 ts_tablet_manager.cc:1939] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 4c6f62c87e1346978aae323f34941727: Deleting consensus metadata
I20260929 15:51:42.281692 20799 catalog_manager.cc:4841] TS 4c6f62c87e1346978aae323f34941727 (127.15.196.194:38567): tablet 4c639bc817fe4ab9be000ea7b8f3a4e3 (table test-table [id=e15318c7fb9849faaa03bf3dbc6dd11e]) successfully deleted
I20260929 15:51:42.386603 21303 tablet_replica.cc:330] T 4c639bc817fe4ab9be000ea7b8f3a4e3 P 137185f289b5460e93cee404d3aea658: stopping tablet replica
I20260929 15:51:42.386852 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 20873
I20260929 15:51:42.404688 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 21004
W20260929 15:51:42.404702 20800 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.196.193:39475 (error 108)
I20260929 15:51:42.406900 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.406321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:39475 (local address 127.0.0.1:35962)
0929 15:51:42.406575 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:42.406713 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:39475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260929 15:51:42.407178 20800 catalog_manager.cc:4584] TS 137185f289b5460e93cee404d3aea658 (127.15.196.193:39475): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4c639bc817fe4ab9be000ea7b8f3a4e3: Network error: Client connection negotiation failed: client connection to 127.15.196.193:39475: connect: Connection refused (error 111)
I20260929 15:51:42.411177 21278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.405830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55146 (local address 127.15.196.194:38567)
0929 15:51:42.406019 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:42.406026 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:42.406051 (+ 25us) server_negotiation.cc:407] Connection header received
0929 15:51:42.406139 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:42.406145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:42.406207 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:42.406317 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:42.410509 (+ 4192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:42.410516 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:42.410522 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:42.410543 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:42.410584 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:42.410600 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:42.410743 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:42.410805 (+ 62us) server_negotiation.cc:299] Negotiation successful
0929 15:51:42.410864 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 15:51:42.412065 21126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:42.460855 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.460394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:39475 (local address 127.0.0.1:35970)
0929 15:51:42.460618 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:42.460710 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:39475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260929 15:51:42.461910 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.461559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:39475 (local address 127.0.0.1:35976)
0929 15:51:42.461707 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:42.461785 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:39475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 15:51:42.522660 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.522091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:39475 (local address 127.0.0.1:35978)
0929 15:51:42.522363 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:42.522483 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:39475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260929 15:51:42.523877 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.523433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:39475 (local address 127.0.0.1:35990)
0929 15:51:42.523619 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:42.523710 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:39475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 15:51:42.528086 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 21004
I20260929 15:51:42.546178 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 21134
I20260929 15:51:42.549229 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.546938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35370 (local address 127.15.196.195:38209)
0929 15:51:42.547148 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:42.547154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:42.547229 (+ 75us) server_negotiation.cc:407] Connection header received
0929 15:51:42.547472 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:42.547476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:42.547549 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:42.547655 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:42.548568 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:42.548574 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:42.548579 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:42.548603 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:42.548634 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:42.548646 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:42.548801 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:42.548883 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 15:51:42.548944 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 15:51:42.549978 21256 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:42.604724 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.604103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:39475 (local address 127.0.0.1:36006)
0929 15:51:42.604422 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:42.604538 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:39475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260929 15:51:42.605882 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:42.605435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:39475 (local address 127.0.0.1:36008)
0929 15:51:42.605652 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:42.605738 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:39475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260929 15:51:42.639588 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 21134
I20260929 15:51:42.656666 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 20783
I20260929 15:51:42.657966 20844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:42.764839 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 20783
2026-09-29T15:51:42Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2721 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260929 15:51:42.805966 16147 mini_oidc.cc:152] Starting JWKS server
I20260929 15:51:42.808642 16147 webserver.cc:460] Webserver started at https://127.0.0.1:37533/ using document root <none> and password file <none>
I20260929 15:51:42.808992 16147 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 15:51:42.809689 16147 webserver.cc:460] Webserver started at http://127.0.0.1:46645/ using document root <none> and password file <none>
2026-09-29T15:51:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:51:42Z Disabled control of system clock
I20260929 15:51:42.836617 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:45283
--webserver_interface=127.15.196.254
--webserver_port=0
--builtin_ntp_servers=127.15.196.212:34837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:45283
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37533/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/testchainca.cert with env {}
W20260929 15:51:43.170425 21316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:43.170727 21316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:43.170917 21316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:43.178279 21316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:43.178387 21316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:43.178481 21316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:43.178596 21316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:43.178673 21316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 15:51:43.178772 21316 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 15:51:43.178829 21316 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37533/jwks.json
I20260929 15:51:43.188923 21316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:34837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:45283
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:45283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37533/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:43.190241 21316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:43.191601 21316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:43.207643 21322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:43.207799 21321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:43.207806 21324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:43.208825 21316 server_base.cc:1034] running on GCE node
I20260929 15:51:43.209723 21316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:43.211421 21316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:43.212628 21316 hybrid_clock.cc:648] HybridClock initialized: now 1790697103212589 us; error 34 us; skew 500 ppm
I20260929 15:51:43.212999 21316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:51:43.215413 21316 webserver.cc:460] Webserver started at http://127.15.196.254:42947/ using document root <none> and password file <none>
I20260929 15:51:43.216125 21316 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:43.216245 21316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:43.216588 21316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:43.219650 21316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "ed6b8e54803e4351b3e6fe4d8d47e7b2"
format_stamp: "Formatted at 2026-09-29 15:51:43 on dist-test-slave-c51v"
server_key: "60f5e2697135307acbdb0a7adcb6ac8f"
server_key_iv: "cbc69df2c7986f2bd8ab70cd087b4626"
server_key_version: "encryptionkey@0"
I20260929 15:51:43.220454 21316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "ed6b8e54803e4351b3e6fe4d8d47e7b2"
format_stamp: "Formatted at 2026-09-29 15:51:43 on dist-test-slave-c51v"
server_key: "60f5e2697135307acbdb0a7adcb6ac8f"
server_key_iv: "cbc69df2c7986f2bd8ab70cd087b4626"
server_key_version: "encryptionkey@0"
I20260929 15:51:43.225499 21316 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:51:43.228874 21330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:43.230208 21316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 15:51:43.230396 21316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "ed6b8e54803e4351b3e6fe4d8d47e7b2"
format_stamp: "Formatted at 2026-09-29 15:51:43 on dist-test-slave-c51v"
server_key: "60f5e2697135307acbdb0a7adcb6ac8f"
server_key_iv: "cbc69df2c7986f2bd8ab70cd087b4626"
server_key_version: "encryptionkey@0"
I20260929 15:51:43.230618 21316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:43.250773 21316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:43.251689 21316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:43.252019 21316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:43.270357 21316 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:45283
I20260929 15:51:43.270377 21381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:45283 every 8 connection(s)
I20260929 15:51:43.271834 21316 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:51:43.274693 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 21316
I20260929 15:51:43.274916 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:51:43.275720 16147 external_mini_cluster.cc:1428] Setting key 4adfc8435b1f1a50e1f12050f69c86a5
I20260929 15:51:43.276042 21382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:43.288293 21382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2: Bootstrap starting.
I20260929 15:51:43.289158 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:43.277665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53336 (local address 127.15.196.254:45283)
0929 15:51:43.278638 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:43.278648 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:43.278689 (+ 41us) server_negotiation.cc:407] Connection header received
0929 15:51:43.280625 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:43.280649 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:43.281120 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:43.281511 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:43.282098 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:43.283408 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:43.284706 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:43.285380 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:43.285611 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:43.286864 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:43.286871 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:43.286884 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:43.287472 (+ 588us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:43.287545 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:43.287586 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:43.287714 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:43.287946 (+ 232us) server_negotiation.cc:299] Negotiation successful
0929 15:51:43.288226 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":366,"threads_started":1}
I20260929 15:51:43.291569 21382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:43.292737 21382 log.cc:824] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:43.292912 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:51:43.295511 21382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2: No bootstrap required, opened a new log
I20260929 15:51:43.302884 21382 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6b8e54803e4351b3e6fe4d8d47e7b2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45283 } }
I20260929 15:51:43.303241 21382 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:43.303380 21382 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed6b8e54803e4351b3e6fe4d8d47e7b2, State: Initialized, Role: FOLLOWER
I20260929 15:51:43.304001 21382 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6b8e54803e4351b3e6fe4d8d47e7b2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45283 } }
I20260929 15:51:43.304229 21382 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:51:43.304361 21382 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:51:43.304534 21382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:43.306672 21382 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6b8e54803e4351b3e6fe4d8d47e7b2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45283 } }
I20260929 15:51:43.307185 21382 leader_election.cc:302] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed6b8e54803e4351b3e6fe4d8d47e7b2; no voters:
I20260929 15:51:43.307829 21382 leader_election.cc:288] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:51:43.308002 21387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:43.308679 21387 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [term 1 LEADER]: Becoming Leader. State: Replica: ed6b8e54803e4351b3e6fe4d8d47e7b2, State: Running, Role: LEADER
I20260929 15:51:43.309252 21387 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6b8e54803e4351b3e6fe4d8d47e7b2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45283 } }
I20260929 15:51:43.310065 21382 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:43.312340 21389 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed6b8e54803e4351b3e6fe4d8d47e7b2. Latest consensus state: current_term: 1 leader_uuid: "ed6b8e54803e4351b3e6fe4d8d47e7b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6b8e54803e4351b3e6fe4d8d47e7b2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45283 } } }
I20260929 15:51:43.312430 21388 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed6b8e54803e4351b3e6fe4d8d47e7b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6b8e54803e4351b3e6fe4d8d47e7b2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45283 } } }
I20260929 15:51:43.312706 21389 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:43.312783 21388 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2 [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:43.316289 21393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:43.321128 21393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:43.329442 21393 catalog_manager.cc:1347] Generated new cluster ID: f98ced451d544228a27c492860005c21
I20260929 15:51:43.329577 21393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:43.352809 21393 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:51:43.354184 21393 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:43.364207 21393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed6b8e54803e4351b3e6fe4d8d47e7b2: Generated new TSK 0
I20260929 15:51:43.365206 21393 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 15:51:43.411119 21412 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260929 15:51:43.411350 21412 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 15:51:43.412462 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:43.390105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53340 (local address 127.15.196.254:45283)
0929 15:51:43.390309 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:43.390316 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:43.390913 (+ 597us) server_negotiation.cc:407] Connection header received
0929 15:51:43.391336 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:43.391349 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:43.391426 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:43.391562 (+ 136us) server_negotiation.cc:226] Negotiated authn=JWT
0929 15:51:43.392210 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:43.392944 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:43.395063 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:43.395571 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:43.395671 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:43.396333 (+ 662us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 15:51:43.411919 (+ 15586us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37533/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 15:51:43.412224 (+ 305us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53340: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37533/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":72}
W20260929 15:51:43.412617 21385 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53340: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37533/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260929 15:51:43.414520 21411 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:51:43.389977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:45283 (local address 127.0.0.1:53340)
0929 15:51:43.390793 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:43.390812 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:43.391019 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:43.391618 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:43.391623 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:43.391636 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0929 15:51:43.392054 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:43.392066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:43.393095 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:43.393100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:43.394906 (+ 1806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:43.394921 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:43.395753 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:43.395761 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:43.395960 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:43.396044 (+ 84us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 15:51:43.412242 (+ 16198us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37533/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 15:51:43.412634 (+ 392us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.196.254:45283: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37533/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":463,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.196.254:45283: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37533/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260929 15:51:43.416086 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 21316
I20260929 15:51:43.417326 21377 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:43.514428 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 21316
2026-09-29T15:51:43Z chronyd exiting
I20260929 15:51:43.565026 16147 test_util.cc:175] -----------------------------------------------
I20260929 15:51:43.565174 16147 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697059131395-16147-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (763 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260929 15:51:43.568972 16147 mini_oidc.cc:152] Starting JWKS server
I20260929 15:51:43.570757 16147 webserver.cc:460] Webserver started at https://127.0.0.1:41021/ using document root <none> and password file <none>
I20260929 15:51:43.570972 16147 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 15:51:43.571602 16147 webserver.cc:460] Webserver started at http://127.0.0.1:44821/ using document root <none> and password file <none>
2026-09-29T15:51:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:51:43Z Disabled control of system clock
I20260929 15:51:43.597733 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:40809
--webserver_interface=127.15.196.254
--webserver_port=0
--builtin_ntp_servers=127.15.196.212:46081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:40809
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41021/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/testchainca.cert with env {}
W20260929 15:51:43.919538 21419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:43.919941 21419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:43.920112 21419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:43.927255 21419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:43.927433 21419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:43.927481 21419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:43.927531 21419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:43.927573 21419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 15:51:43.927634 21419 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 15:51:43.927675 21419 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41021/jwks.json
I20260929 15:51:43.937572 21419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:46081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:40809
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:40809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41021/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:43.938802 21419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:43.940165 21419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:43.952216 21427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:43.952147 21425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:43.952174 21424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:43.953154 21419 server_base.cc:1034] running on GCE node
I20260929 15:51:43.953960 21419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:43.955479 21419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:43.956707 21419 hybrid_clock.cc:648] HybridClock initialized: now 1790697103956657 us; error 45 us; skew 500 ppm
I20260929 15:51:43.957060 21419 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:51:43.959384 21419 webserver.cc:460] Webserver started at http://127.15.196.254:34381/ using document root <none> and password file <none>
I20260929 15:51:43.960090 21419 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:43.960224 21419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:43.960559 21419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:43.963559 21419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "454b7b9a21b9453c927af356eace5fd2"
format_stamp: "Formatted at 2026-09-29 15:51:43 on dist-test-slave-c51v"
server_key: "dbdd5d1c025df5d3c1aad581234ea24e"
server_key_iv: "df575a84cde58c5e073e999bac272746"
server_key_version: "encryptionkey@0"
I20260929 15:51:43.964344 21419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "454b7b9a21b9453c927af356eace5fd2"
format_stamp: "Formatted at 2026-09-29 15:51:43 on dist-test-slave-c51v"
server_key: "dbdd5d1c025df5d3c1aad581234ea24e"
server_key_iv: "df575a84cde58c5e073e999bac272746"
server_key_version: "encryptionkey@0"
I20260929 15:51:43.969365 21419 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:51:43.972715 21433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:43.974120 21419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 15:51:43.974308 21419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "454b7b9a21b9453c927af356eace5fd2"
format_stamp: "Formatted at 2026-09-29 15:51:43 on dist-test-slave-c51v"
server_key: "dbdd5d1c025df5d3c1aad581234ea24e"
server_key_iv: "df575a84cde58c5e073e999bac272746"
server_key_version: "encryptionkey@0"
I20260929 15:51:43.974532 21419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:43.985625 21419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:43.986444 21419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:43.986755 21419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:44.004660 21419 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:40809
I20260929 15:51:44.004664 21484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:40809 every 8 connection(s)
I20260929 15:51:44.006054 21419 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:51:44.009835 21485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:44.015820 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 21419
I20260929 15:51:44.016036 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:51:44.016687 16147 external_mini_cluster.cc:1428] Setting key f1f777362877dff9eb80ffab09648864
I20260929 15:51:44.021282 21485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2: Bootstrap starting.
I20260929 15:51:44.024475 21485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:44.025480 21485 log.cc:824] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:44.028522 21485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2: No bootstrap required, opened a new log
I20260929 15:51:44.030345 21488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:44.018655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36208 (local address 127.15.196.254:40809)
0929 15:51:44.019988 (+ 1333us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:44.019999 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:44.020042 (+ 43us) server_negotiation.cc:407] Connection header received
0929 15:51:44.021730 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:44.021753 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:44.022216 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:44.022614 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:44.023197 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:44.024562 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:44.025904 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:44.026627 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:44.026816 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:44.028039 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:44.028046 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:44.028060 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:44.028593 (+ 533us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:44.028711 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:44.028733 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:44.028848 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:44.029101 (+ 253us) server_negotiation.cc:299] Negotiation successful
0929 15:51:44.029416 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":772,"thread_start_us":414,"threads_started":1}
I20260929 15:51:44.034478 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:51:44.037267 21485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454b7b9a21b9453c927af356eace5fd2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40809 } }
I20260929 15:51:44.037746 21485 raft_consensus.cc:374] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:44.037856 21485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 454b7b9a21b9453c927af356eace5fd2, State: Initialized, Role: FOLLOWER
I20260929 15:51:44.038374 21485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454b7b9a21b9453c927af356eace5fd2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40809 } }
I20260929 15:51:44.038581 21485 raft_consensus.cc:388] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:51:44.038674 21485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:51:44.038796 21485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:44.041371 21485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454b7b9a21b9453c927af356eace5fd2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40809 } }
I20260929 15:51:44.041826 21485 leader_election.cc:302] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 454b7b9a21b9453c927af356eace5fd2; no voters:
I20260929 15:51:44.042341 21485 leader_election.cc:288] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:51:44.042496 21490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:44.043219 21490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [term 1 LEADER]: Becoming Leader. State: Replica: 454b7b9a21b9453c927af356eace5fd2, State: Running, Role: LEADER
I20260929 15:51:44.043720 21490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454b7b9a21b9453c927af356eace5fd2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40809 } }
I20260929 15:51:44.044473 21485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:44.046736 21491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "454b7b9a21b9453c927af356eace5fd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454b7b9a21b9453c927af356eace5fd2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40809 } } }
I20260929 15:51:44.047129 21491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:44.046849 21492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 454b7b9a21b9453c927af356eace5fd2. Latest consensus state: current_term: 1 leader_uuid: "454b7b9a21b9453c927af356eace5fd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454b7b9a21b9453c927af356eace5fd2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40809 } } }
I20260929 15:51:44.047273 21492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2 [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:44.050019 21496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:44.057168 21496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:44.066210 21496 catalog_manager.cc:1347] Generated new cluster ID: b5785394cf6d41189e3d477ed511a912
I20260929 15:51:44.066339 21496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:44.079049 21496 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:51:44.081089 21496 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:44.118868 21496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 454b7b9a21b9453c927af356eace5fd2: Generated new TSK 0
I20260929 15:51:44.119980 21496 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 15:51:44.158046 21515 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260929 15:51:44.158174 21515 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 15:51:44.159433 21488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:44.139017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36210 (local address 127.15.196.254:40809)
0929 15:51:44.139203 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:44.139209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:44.139575 (+ 366us) server_negotiation.cc:407] Connection header received
0929 15:51:44.139785 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:44.139791 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:44.139891 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:44.140062 (+ 171us) server_negotiation.cc:226] Negotiated authn=JWT
0929 15:51:44.140486 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:44.141641 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:44.142976 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:44.143758 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:44.143879 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:44.144378 (+ 499us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 15:51:44.158747 (+ 14369us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41021/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 15:51:44.159125 (+ 378us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36210: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41021/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":71}
W20260929 15:51:44.159570 21488 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36210: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41021/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260929 15:51:44.159619 21514 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:51:44.138828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:40809 (local address 127.0.0.1:36210)
0929 15:51:44.139470 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:44.139486 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:44.139605 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:44.140076 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:44.140079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:44.140087 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0929 15:51:44.140326 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:44.140334 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:44.141782 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:44.141786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:44.142836 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:44.142846 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:44.143942 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:44.143946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:44.144098 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:44.144131 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 15:51:44.158998 (+ 14867us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41021/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 15:51:44.159187 (+ 189us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.196.254:40809: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41021/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":343,"threads_started":1}
I20260929 15:51:44.160744 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 21419
I20260929 15:51:44.162070 21480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:44.258764 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 21419
2026-09-29T15:51:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (745 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260929 15:51:44.314652 16147 mini_oidc.cc:152] Starting JWKS server
I20260929 15:51:44.316553 16147 webserver.cc:460] Webserver started at https://127.0.0.1:35559/ using document root <none> and password file <none>
I20260929 15:51:44.316828 16147 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 15:51:44.317442 16147 webserver.cc:460] Webserver started at http://127.0.0.1:39565/ using document root <none> and password file <none>
2026-09-29T15:51:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:51:44Z Disabled control of system clock
I20260929 15:51:44.344321 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:37981
--webserver_interface=127.15.196.254
--webserver_port=0
--builtin_ntp_servers=127.15.196.212:42353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:37981
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35559/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 15:51:44.667590 21522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:44.667874 21522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:44.667959 21522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:44.675048 21522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:44.675151 21522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:44.675190 21522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:44.675246 21522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:44.675305 21522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 15:51:44.675417 21522 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 15:51:44.675467 21522 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35559/jwks.json
I20260929 15:51:44.684702 21522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:42353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:37981
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:37981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35559/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:44.685938 21522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:44.687772 21522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:44.699607 21527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:44.699648 21528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:44.700263 21530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:44.700379 21522 server_base.cc:1034] running on GCE node
I20260929 15:51:44.701164 21522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:44.702651 21522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:44.703876 21522 hybrid_clock.cc:648] HybridClock initialized: now 1790697104703822 us; error 50 us; skew 500 ppm
I20260929 15:51:44.704239 21522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:51:44.706578 21522 webserver.cc:460] Webserver started at http://127.15.196.254:46487/ using document root <none> and password file <none>
I20260929 15:51:44.707363 21522 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:44.707500 21522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:44.707829 21522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:44.710744 21522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "8a691f34f768417a89dd66de10f2ec2f"
format_stamp: "Formatted at 2026-09-29 15:51:44 on dist-test-slave-c51v"
server_key: "dca8caab20b832fdcd35bf0d341da6d3"
server_key_iv: "56bdc058145c7c5c91c88ba8fb17aa91"
server_key_version: "encryptionkey@0"
I20260929 15:51:44.711673 21522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "8a691f34f768417a89dd66de10f2ec2f"
format_stamp: "Formatted at 2026-09-29 15:51:44 on dist-test-slave-c51v"
server_key: "dca8caab20b832fdcd35bf0d341da6d3"
server_key_iv: "56bdc058145c7c5c91c88ba8fb17aa91"
server_key_version: "encryptionkey@0"
I20260929 15:51:44.717056 21522 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260929 15:51:44.720625 21536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:44.721990 21522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:51:44.722167 21522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "8a691f34f768417a89dd66de10f2ec2f"
format_stamp: "Formatted at 2026-09-29 15:51:44 on dist-test-slave-c51v"
server_key: "dca8caab20b832fdcd35bf0d341da6d3"
server_key_iv: "56bdc058145c7c5c91c88ba8fb17aa91"
server_key_version: "encryptionkey@0"
I20260929 15:51:44.722380 21522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:44.747210 21522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:44.748140 21522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:44.748481 21522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:44.766959 21522 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:37981
I20260929 15:51:44.766992 21587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:37981 every 8 connection(s)
I20260929 15:51:44.768447 21522 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:51:44.772778 21588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:44.773814 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 21522
I20260929 15:51:44.774044 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:51:44.774890 16147 external_mini_cluster.cc:1428] Setting key f682e0810a9218d7e71f95271e378cf9
I20260929 15:51:44.785933 21588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f: Bootstrap starting.
I20260929 15:51:44.788203 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:44.776968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46318 (local address 127.15.196.254:37981)
0929 15:51:44.778014 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:44.778026 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:44.778067 (+ 41us) server_negotiation.cc:407] Connection header received
0929 15:51:44.779749 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:44.779771 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:44.780247 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:44.780657 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:44.781187 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:44.782432 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:44.783813 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:44.784504 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:44.784714 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:44.785899 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:44.785907 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:44.785924 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:44.786516 (+ 592us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:44.786590 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:44.786611 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:44.786735 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:44.786955 (+ 220us) server_negotiation.cc:299] Negotiation successful
0929 15:51:44.787239 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":361,"threads_started":1}
I20260929 15:51:44.789191 21588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:44.790210 21588 log.cc:824] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:44.791903 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:51:44.793447 21588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f: No bootstrap required, opened a new log
I20260929 15:51:44.801071 21588 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a691f34f768417a89dd66de10f2ec2f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 37981 } }
I20260929 15:51:44.801394 21588 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:44.801476 21588 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a691f34f768417a89dd66de10f2ec2f, State: Initialized, Role: FOLLOWER
I20260929 15:51:44.802013 21588 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a691f34f768417a89dd66de10f2ec2f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 37981 } }
I20260929 15:51:44.802205 21588 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:51:44.802299 21588 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:51:44.802410 21588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:44.804670 21588 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a691f34f768417a89dd66de10f2ec2f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 37981 } }
I20260929 15:51:44.805255 21588 leader_election.cc:302] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a691f34f768417a89dd66de10f2ec2f; no voters:
I20260929 15:51:44.805853 21588 leader_election.cc:288] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:51:44.806021 21593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:44.806717 21593 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [term 1 LEADER]: Becoming Leader. State: Replica: 8a691f34f768417a89dd66de10f2ec2f, State: Running, Role: LEADER
I20260929 15:51:44.807402 21593 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a691f34f768417a89dd66de10f2ec2f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 37981 } }
I20260929 15:51:44.808044 21588 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:44.810415 21594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a691f34f768417a89dd66de10f2ec2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a691f34f768417a89dd66de10f2ec2f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 37981 } } }
I20260929 15:51:44.810786 21594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:44.811635 21602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:44.810400 21595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a691f34f768417a89dd66de10f2ec2f. Latest consensus state: current_term: 1 leader_uuid: "8a691f34f768417a89dd66de10f2ec2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a691f34f768417a89dd66de10f2ec2f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 37981 } } }
I20260929 15:51:44.812352 21595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:44.818027 21602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:44.826905 21602 catalog_manager.cc:1347] Generated new cluster ID: 57051124cff34c7a9619c5b235e986af
I20260929 15:51:44.827026 21602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:44.853247 21602 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:51:44.855054 21602 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:44.874382 21602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a691f34f768417a89dd66de10f2ec2f: Generated new TSK 0
I20260929 15:51:44.875552 21602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:51:44.913407 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:44.904725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46322 (local address 127.15.196.254:37981)
0929 15:51:44.904926 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:44.904932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:44.905277 (+ 345us) server_negotiation.cc:407] Connection header received
0929 15:51:44.905501 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:44.905507 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:44.905605 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:44.905778 (+ 173us) server_negotiation.cc:226] Negotiated authn=JWT
0929 15:51:44.906195 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:44.907445 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:44.908806 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:44.909581 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:44.909715 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:44.910217 (+ 502us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 15:51:44.912754 (+ 2537us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35559/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 15:51:44.913112 (+ 358us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46322: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35559/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":74}
W20260929 15:51:44.913553 21591 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46322: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35559/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260929 15:51:44.915143 21617 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:51:44.904580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:37981 (local address 127.0.0.1:46322)
0929 15:51:44.905174 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:44.905189 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:44.905310 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:44.905786 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:44.905789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:44.905798 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0929 15:51:44.906059 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:44.906066 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:44.907590 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:44.907594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:44.908674 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:44.908686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:44.909756 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:44.909762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:44.909890 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:44.909923 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 15:51:44.912963 (+ 3040us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35559/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 15:51:44.914733 (+ 1770us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.196.254:37981: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35559/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":1412,"thread_start_us":315,"threads_started":1}
I20260929 15:51:44.916400 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 21522
I20260929 15:51:44.917639 21583 generic_service.cc:196] Checking for leaks (request via RPC)
E20260929 15:51:44.925932 21618 webserver.cc:533] Webserver: sslize failed:
E20260929 15:51:44.926024 21618 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 15:51:45.016052 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 21522
2026-09-29T15:51:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (757 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-29T15:51:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:51:45Z Disabled control of system clock
I20260929 15:51:45.097131 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:44127
--webserver_interface=127.15.196.254
--webserver_port=0
--builtin_ntp_servers=127.15.196.212:46797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:44127
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/testchainca.cert with env {}
W20260929 15:51:45.421320 21623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:45.421649 21623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:45.421832 21623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:45.429217 21623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:45.429324 21623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:45.429361 21623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:45.429507 21623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:45.429597 21623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 15:51:45.429706 21623 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 15:51:45.429781 21623 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260929 15:51:45.439918 21623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:46797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:44127
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:44127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:45.441262 21623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:45.442622 21623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:45.454241 21628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:45.454201 21631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:45.454282 21629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:45.454933 21623 server_base.cc:1034] running on GCE node
I20260929 15:51:45.455789 21623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:45.457172 21623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:45.458423 21623 hybrid_clock.cc:648] HybridClock initialized: now 1790697105458371 us; error 42 us; skew 500 ppm
I20260929 15:51:45.458781 21623 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:51:45.461179 21623 webserver.cc:460] Webserver started at http://127.15.196.254:32871/ using document root <none> and password file <none>
I20260929 15:51:45.461918 21623 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:45.462055 21623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:45.462364 21623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:45.465265 21623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "103e50b966d44635a56b612820e377a7"
format_stamp: "Formatted at 2026-09-29 15:51:45 on dist-test-slave-c51v"
server_key: "38ad5fd3f47a76e3bfa34bc9c9ac97d9"
server_key_iv: "e016af5638bd3c7856c063b94cc805c5"
server_key_version: "encryptionkey@0"
I20260929 15:51:45.466064 21623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "103e50b966d44635a56b612820e377a7"
format_stamp: "Formatted at 2026-09-29 15:51:45 on dist-test-slave-c51v"
server_key: "38ad5fd3f47a76e3bfa34bc9c9ac97d9"
server_key_iv: "e016af5638bd3c7856c063b94cc805c5"
server_key_version: "encryptionkey@0"
I20260929 15:51:45.471356 21623 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 15:51:45.474948 21637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:45.476481 21623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:45.476665 21623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "103e50b966d44635a56b612820e377a7"
format_stamp: "Formatted at 2026-09-29 15:51:45 on dist-test-slave-c51v"
server_key: "38ad5fd3f47a76e3bfa34bc9c9ac97d9"
server_key_iv: "e016af5638bd3c7856c063b94cc805c5"
server_key_version: "encryptionkey@0"
I20260929 15:51:45.476893 21623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:45.493184 21623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:45.494100 21623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:45.494405 21623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:45.513123 21623 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:44127
I20260929 15:51:45.513140 21688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:44127 every 8 connection(s)
I20260929 15:51:45.514552 21623 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:51:45.515662 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 21623
I20260929 15:51:45.515870 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:51:45.516698 16147 external_mini_cluster.cc:1428] Setting key 128775f9de505cc9958961e3e386bdf3
I20260929 15:51:45.519726 21689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:45.531720 21692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:45.518719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43604 (local address 127.15.196.254:44127)
0929 15:51:45.519867 (+ 1148us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:45.519881 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:45.519924 (+ 43us) server_negotiation.cc:407] Connection header received
0929 15:51:45.521536 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:45.521559 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:45.522038 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:45.522475 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:45.523074 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:45.524498 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:45.525855 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:45.526752 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:45.526944 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:45.528668 (+ 1724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:45.528678 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:45.528703 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:51:45.529483 (+ 780us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:45.529589 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:51:45.529622 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:45.529764 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:45.529983 (+ 219us) server_negotiation.cc:299] Negotiation successful
0929 15:51:45.530345 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":344,"threads_started":1}
I20260929 15:51:45.532387 21689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7: Bootstrap starting.
I20260929 15:51:45.535684 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:51:45.536019 21689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:45.537036 21689 log.cc:824] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:45.540084 21689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7: No bootstrap required, opened a new log
I20260929 15:51:45.547689 21689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "103e50b966d44635a56b612820e377a7" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44127 } }
I20260929 15:51:45.548126 21689 raft_consensus.cc:374] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:45.548235 21689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 103e50b966d44635a56b612820e377a7, State: Initialized, Role: FOLLOWER
I20260929 15:51:45.548800 21689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "103e50b966d44635a56b612820e377a7" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44127 } }
I20260929 15:51:45.549044 21689 raft_consensus.cc:388] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:51:45.549188 21689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:51:45.549363 21689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:45.551796 21689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "103e50b966d44635a56b612820e377a7" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44127 } }
I20260929 15:51:45.552413 21689 leader_election.cc:302] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 103e50b966d44635a56b612820e377a7; no voters:
I20260929 15:51:45.553104 21689 leader_election.cc:288] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:51:45.553200 21694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:45.554338 21694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [term 1 LEADER]: Becoming Leader. State: Replica: 103e50b966d44635a56b612820e377a7, State: Running, Role: LEADER
I20260929 15:51:45.554867 21694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "103e50b966d44635a56b612820e377a7" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44127 } }
I20260929 15:51:45.555152 21689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:45.558307 21696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 103e50b966d44635a56b612820e377a7. Latest consensus state: current_term: 1 leader_uuid: "103e50b966d44635a56b612820e377a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "103e50b966d44635a56b612820e377a7" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44127 } } }
I20260929 15:51:45.558622 21696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:45.559213 21695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "103e50b966d44635a56b612820e377a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "103e50b966d44635a56b612820e377a7" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44127 } } }
I20260929 15:51:45.559544 21695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7 [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:45.559522 21700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:45.565274 21700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:45.574254 21700 catalog_manager.cc:1347] Generated new cluster ID: 480fa8e8b3ce4702b40da8c039fca95d
I20260929 15:51:45.574402 21700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:45.616367 21700 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:51:45.617749 21700 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:45.626431 21700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 103e50b966d44635a56b612820e377a7: Generated new TSK 0
I20260929 15:51:45.627596 21700 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:51:45.683372 21692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:45.655134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43614 (local address 127.15.196.254:44127)
0929 15:51:45.655388 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:45.655397 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:45.655890 (+ 493us) server_negotiation.cc:407] Connection header received
0929 15:51:45.656197 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:45.656202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:45.656270 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:45.656435 (+ 165us) server_negotiation.cc:226] Negotiated authn=JWT
0929 15:51:45.657010 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:45.657736 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:45.660099 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:45.660559 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:45.660690 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:45.661335 (+ 645us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 15:51:45.682678 (+ 21343us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 15:51:45.683090 (+ 412us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43614: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":68}
W20260929 15:51:45.683554 21692 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43614: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260929 15:51:45.683888 21718 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:51:45.655055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44127 (local address 127.0.0.1:43614)
0929 15:51:45.655782 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:45.655803 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:45.655965 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:45.656475 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:45.656482 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:45.656496 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0929 15:51:45.656836 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:45.656849 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:45.657899 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:45.657906 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:45.659945 (+ 2039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:45.659964 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:45.660769 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:45.660776 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:45.661001 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:45.661055 (+ 54us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 15:51:45.683048 (+ 21993us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 15:51:45.683448 (+ 400us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.196.254:44127: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":397,"threads_started":1}
I20260929 15:51:45.685441 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 21623
I20260929 15:51:45.686694 21684 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:45.781873 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 21623
2026-09-29T15:51:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (744 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (414 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (405 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:51:46 dist-test-slave-c51v krb5kdc[21733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:51:46 dist-test-slave-c51v krb5kdc[21733](info): set up 2 sockets
Sep 29 15:51:46 dist-test-slave-c51v krb5kdc[21733](info): commencing operation
krb5kdc: starting...
W20260929 15:51:47.693882 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:51:47 dist-test-slave-c51v krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:51:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:51:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:51:47.903944 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:40931
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:34283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:40931
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:48.236570 21746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:48.236938 21746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:48.237108 21746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:48.244637 21746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:48.244791 21746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:48.244923 21746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:48.245030 21746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:48.245126 21746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:51:48.255317 21746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:34283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:40931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:40931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:48.256708 21746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:48.258148 21746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:48.270740 21754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:48.270852 21751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:48.270854 21752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:48.271867 21746 server_base.cc:1034] running on GCE node
I20260929 15:51:48.272658 21746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:48.274073 21746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:48.275353 21746 hybrid_clock.cc:648] HybridClock initialized: now 1790697108275246 us; error 64 us; skew 500 ppm
I20260929 15:51:48.275717 21746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:48 dist-test-slave-c51v krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697108, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:48.283200 21746 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:51:48.284811 21746 webserver.cc:460] Webserver started at http://127.15.196.254:46573/ using document root <none> and password file <none>
I20260929 15:51:48.285485 21746 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:48.285583 21746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:48.285835 21746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:48.288846 21746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "28c1d8a908684896b15f34a567e44753"
format_stamp: "Formatted at 2026-09-29 15:51:48 on dist-test-slave-c51v"
server_key: "f132ec1ed3060661f35bad29feb652d2"
server_key_iv: "04e587d5ee410ca9e7c82c403024298f"
server_key_version: "encryptionkey@0"
I20260929 15:51:48.289574 21746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "28c1d8a908684896b15f34a567e44753"
format_stamp: "Formatted at 2026-09-29 15:51:48 on dist-test-slave-c51v"
server_key: "f132ec1ed3060661f35bad29feb652d2"
server_key_iv: "04e587d5ee410ca9e7c82c403024298f"
server_key_version: "encryptionkey@0"
I20260929 15:51:48.294651 21746 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 15:51:48.298146 21761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:48.299639 21746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 15:51:48.299813 21746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "28c1d8a908684896b15f34a567e44753"
format_stamp: "Formatted at 2026-09-29 15:51:48 on dist-test-slave-c51v"
server_key: "f132ec1ed3060661f35bad29feb652d2"
server_key_iv: "04e587d5ee410ca9e7c82c403024298f"
server_key_version: "encryptionkey@0"
I20260929 15:51:48.300019 21746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:48.317775 21746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:48.322062 21746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:48.322384 21746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:48.341176 21746 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:40931
I20260929 15:51:48.341199 21812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:40931 every 8 connection(s)
I20260929 15:51:48.342582 21746 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:51:48.343470 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 21746
I20260929 15:51:48.343700 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:51:48.344381 16147 external_mini_cluster.cc:1428] Setting key db18c634f92c2c4bd9718703d49c78f8
I20260929 15:51:48.346984 21813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 15:51:48 dist-test-slave-c51v krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:48.359987 21813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753: Bootstrap starting.
I20260929 15:51:48.363080 21813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:48.364174 21813 log.cc:824] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:48.367238 21813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753: No bootstrap required, opened a new log
I20260929 15:51:48.367532 21816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:48.346371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34836 (local address 127.15.196.254:40931)
0929 15:51:48.347213 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:48.347226 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:48.347265 (+ 39us) server_negotiation.cc:407] Connection header received
0929 15:51:48.349026 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:48.349048 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:48.349438 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:48.349769 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:48.351552 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:48.352877 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:48.354617 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:48.355243 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:48.355442 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:48.360137 (+ 4695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:48.360148 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:48.360170 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:48.360218 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:48.363240 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:48.363985 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:48.363994 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:48.364013 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:48.364173 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:48.364556 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:48.364562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:48.364566 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:48.365354 (+ 788us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:48.365609 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:48.366120 (+ 511us) server_negotiation.cc:299] Negotiation successful
0929 15:51:48.366499 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":365,"threads_started":1}
I20260929 15:51:48.375228 21813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c1d8a908684896b15f34a567e44753" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40931 } }
I20260929 15:51:48.375667 21813 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:48.375775 21813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28c1d8a908684896b15f34a567e44753, State: Initialized, Role: FOLLOWER
I20260929 15:51:48.376302 21813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c1d8a908684896b15f34a567e44753" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40931 } }
I20260929 15:51:48.376544 21813 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:51:48.376684 21813 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:51:48.376850 21813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:48.378919 21813 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c1d8a908684896b15f34a567e44753" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40931 } }
I20260929 15:51:48.379503 21813 leader_election.cc:302] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28c1d8a908684896b15f34a567e44753; no voters:
I20260929 15:51:48.380151 21813 leader_election.cc:288] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:51:48.380288 21818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:48.380995 21818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [term 1 LEADER]: Becoming Leader. State: Replica: 28c1d8a908684896b15f34a567e44753, State: Running, Role: LEADER
I20260929 15:51:48.381505 21818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c1d8a908684896b15f34a567e44753" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40931 } }
I20260929 15:51:48.382231 21813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:48.384985 21819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28c1d8a908684896b15f34a567e44753" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c1d8a908684896b15f34a567e44753" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40931 } } }
I20260929 15:51:48.384934 21820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28c1d8a908684896b15f34a567e44753. Latest consensus state: current_term: 1 leader_uuid: "28c1d8a908684896b15f34a567e44753" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c1d8a908684896b15f34a567e44753" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40931 } } }
I20260929 15:51:48.385357 21820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:48.385357 21819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753 [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:48.387827 21826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:48.392853 21826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:48.401438 21826 catalog_manager.cc:1347] Generated new cluster ID: 783a7c46c5874d5ebdace0d1b4297296
I20260929 15:51:48.401597 21826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:48.432655 21826 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:51:48.434082 21826 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:48.444574 21826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28c1d8a908684896b15f34a567e44753: Generated new TSK 0
I20260929 15:51:48.445653 21826 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:51:48.505154 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:40931
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:34283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:48.829397 21839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:48.829680 21839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:48.829773 21839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:48.837075 21839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:48.837235 21839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:48.837374 21839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:51:48.847051 21839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:34283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:40931
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:48.848412 21839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:48.849733 21839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:48.862685 21845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:48.862818 21844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:48.862811 21847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:48.863844 21839 server_base.cc:1034] running on GCE node
I20260929 15:51:48.864382 21839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:48.865257 21839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:48.866489 21839 hybrid_clock.cc:648] HybridClock initialized: now 1790697108866418 us; error 35 us; skew 500 ppm
I20260929 15:51:48.866755 21839 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:48 dist-test-slave-c51v krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697108, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:48.869838 21839 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:51:48.871230 21839 webserver.cc:460] Webserver started at http://127.15.196.193:44459/ using document root <none> and password file <none>
I20260929 15:51:48.871934 21839 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:48.872045 21839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:48.872300 21839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:48.875129 21839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "73e1a6d5c793441e91151d31fb6ff3e0"
format_stamp: "Formatted at 2026-09-29 15:51:48 on dist-test-slave-c51v"
server_key: "602fd50b2c65d1c97256b9e0c94b4970"
server_key_iv: "b837aa7c8b11d402719fa969a769cf43"
server_key_version: "encryptionkey@0"
I20260929 15:51:48.875942 21839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "73e1a6d5c793441e91151d31fb6ff3e0"
format_stamp: "Formatted at 2026-09-29 15:51:48 on dist-test-slave-c51v"
server_key: "602fd50b2c65d1c97256b9e0c94b4970"
server_key_iv: "b837aa7c8b11d402719fa969a769cf43"
server_key_version: "encryptionkey@0"
I20260929 15:51:48.881093 21839 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 15:51:48.884698 21854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:48.886060 21839 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:51:48.886245 21839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "73e1a6d5c793441e91151d31fb6ff3e0"
format_stamp: "Formatted at 2026-09-29 15:51:48 on dist-test-slave-c51v"
server_key: "602fd50b2c65d1c97256b9e0c94b4970"
server_key_iv: "b837aa7c8b11d402719fa969a769cf43"
server_key_version: "encryptionkey@0"
I20260929 15:51:48.886471 21839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:48.905829 21839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:48.910220 21839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:48.910512 21839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:48.911533 21839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:48.912953 21839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:48.913024 21839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:48.913188 21839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:48.913246 21839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:48.948261 21839 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:34535
I20260929 15:51:48.948297 21966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:34535 every 8 connection(s)
I20260929 15:51:48.949539 21839 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:51:48.955343 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 21839
I20260929 15:51:48.955554 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:51:48.956192 16147 external_mini_cluster.cc:1428] Setting key 4a05ff21064ffbe3587c93cae361635a
Sep 29 15:51:48 dist-test-slave-c51v krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697108, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:48.969075 21969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:48.951919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:56471 (local address 127.15.196.254:40931)
0929 15:51:48.952346 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:48.952351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:48.953231 (+ 880us) server_negotiation.cc:407] Connection header received
0929 15:51:48.955044 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:48.955049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:48.955139 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:48.955255 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:48.956937 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:48.957691 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:48.959271 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:48.959732 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:48.959879 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:48.963353 (+ 3474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:48.963359 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:48.963362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:48.963401 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:48.965938 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:48.966686 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:48.966694 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:48.966699 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:48.966824 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:48.967245 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:48.967252 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:48.967255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:48.967585 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:48.967761 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:48.968469 (+ 708us) server_negotiation.cc:299] Negotiation successful
0929 15:51:48.968706 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":229,"threads_started":1}
I20260929 15:51:48.970054 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:48.952365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:40931 (local address 127.15.196.193:56471)
0929 15:51:48.953061 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:48.953109 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:48.954772 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:48.955572 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:48.955581 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:48.956135 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:48.956758 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:48.956775 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:48.957854 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:48.957865 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:48.959112 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:48.959124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:48.959970 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:48.959979 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:48.960211 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:48.960903 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:48.960935 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:48.963078 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:48.966108 (+ 3030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:48.966118 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:48.966136 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:48.966563 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:48.966940 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:48.966946 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:48.966950 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:48.967139 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:48.967720 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:48.967730 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:48.968089 (+ 359us) client_negotiation.cc:769] Sending connection context
0929 15:51:48.968374 (+ 285us) client_negotiation.cc:240] Negotiation successful
0929 15:51:48.968936 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":263,"threads_started":1}
I20260929 15:51:48.971778 21967 heartbeater.cc:347] Connected to a master server at 127.15.196.254:40931
I20260929 15:51:48.972110 21967 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:48.973074 21967 heartbeater.cc:511] Master 127.15.196.254:40931 requested a full tablet report, sending...
I20260929 15:51:48.975522 21778 ts_manager.cc:194] Registered new tserver with Master: 73e1a6d5c793441e91151d31fb6ff3e0 (127.15.196.193:34535)
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
I20260929 15:51:48.976998 21778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:56471
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:51:49.002477 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:40931
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:34283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:49.329949 21973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:49.330276 21973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:49.330451 21973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:49.337651 21973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:49.337800 21973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:49.337971 21973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:51:49.347640 21973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:34283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:40931
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:49.348950 21973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:49.350234 21973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:49.363041 21978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:49.363096 21981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:49.363137 21979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:49.363881 21973 server_base.cc:1034] running on GCE node
I20260929 15:51:49.364506 21973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:49.365578 21973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:49.366818 21973 hybrid_clock.cc:648] HybridClock initialized: now 1790697109366769 us; error 41 us; skew 500 ppm
I20260929 15:51:49.367146 21973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:49 dist-test-slave-c51v krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697109, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:49.370404 21973 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:51:49.372014 21973 webserver.cc:460] Webserver started at http://127.15.196.194:35317/ using document root <none> and password file <none>
I20260929 15:51:49.372751 21973 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:49.372900 21973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:49.373216 21973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:49.376262 21973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "f8ef2910a313445bacb6a5002c00f33a"
format_stamp: "Formatted at 2026-09-29 15:51:49 on dist-test-slave-c51v"
server_key: "d6dcdba8f80346de883e649008b6af4b"
server_key_iv: "7afd50aa6e9ccbc94297f4b1355515ff"
server_key_version: "encryptionkey@0"
I20260929 15:51:49.377066 21973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "f8ef2910a313445bacb6a5002c00f33a"
format_stamp: "Formatted at 2026-09-29 15:51:49 on dist-test-slave-c51v"
server_key: "d6dcdba8f80346de883e649008b6af4b"
server_key_iv: "7afd50aa6e9ccbc94297f4b1355515ff"
server_key_version: "encryptionkey@0"
I20260929 15:51:49.382288 21973 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 15:51:49.385852 21988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:49.387140 21973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:51:49.387372 21973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "f8ef2910a313445bacb6a5002c00f33a"
format_stamp: "Formatted at 2026-09-29 15:51:49 on dist-test-slave-c51v"
server_key: "d6dcdba8f80346de883e649008b6af4b"
server_key_iv: "7afd50aa6e9ccbc94297f4b1355515ff"
server_key_version: "encryptionkey@0"
I20260929 15:51:49.387635 21973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:49.404362 21973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:49.408264 21973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:49.408562 21973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:49.409502 21973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:49.410825 21973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:49.410933 21973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:49.411089 21973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:49.411160 21973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:49.443857 21973 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:40617
I20260929 15:51:49.443902 22100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:40617 every 8 connection(s)
I20260929 15:51:49.445349 21973 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:51:49.451841 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 21973
I20260929 15:51:49.452024 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:51:49.452593 16147 external_mini_cluster.cc:1428] Setting key fcf6f182d2296cf4a2144eba229c8561
Sep 29 15:51:49 dist-test-slave-c51v krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697109, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:49.467411 21969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:49.448094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:58837 (local address 127.15.196.254:40931)
0929 15:51:49.448265 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:49.448270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:49.449488 (+ 1218us) server_negotiation.cc:407] Connection header received
0929 15:51:49.451271 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:49.451275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:49.451412 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:49.451499 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:49.453520 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:49.454480 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:49.456109 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:49.456895 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:49.457040 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:49.461014 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:49.461022 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:49.461029 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:49.461072 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:49.463985 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:49.464904 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:49.464908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:49.464910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:49.465046 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:49.465541 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:49.465544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:49.465546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:49.465799 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:49.465906 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:49.466782 (+ 876us) server_negotiation.cc:299] Negotiation successful
0929 15:51:49.467068 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 15:51:49.468904 22103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:49.448618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:40931 (local address 127.15.196.194:58837)
0929 15:51:49.449330 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:49.449372 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:49.450945 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:49.451789 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:49.451801 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:49.452563 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:49.453338 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:49.453359 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:49.454663 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:49.454667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:49.455957 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:49.455968 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:49.457106 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:49.457110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:49.457305 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:49.458091 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:49.458131 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:49.460771 (+ 2640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:49.464121 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:49.464133 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:49.464153 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:49.464782 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:49.465154 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:49.465161 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:49.465167 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:49.465429 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:49.465929 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:49.465939 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:49.466399 (+ 460us) client_negotiation.cc:769] Sending connection context
0929 15:51:49.466716 (+ 317us) client_negotiation.cc:240] Negotiation successful
0929 15:51:49.467388 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":300,"threads_started":1}
I20260929 15:51:49.470371 22101 heartbeater.cc:347] Connected to a master server at 127.15.196.254:40931
I20260929 15:51:49.470726 22101 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:49.471563 22101 heartbeater.cc:511] Master 127.15.196.254:40931 requested a full tablet report, sending...
I20260929 15:51:49.473095 21778 ts_manager.cc:194] Registered new tserver with Master: f8ef2910a313445bacb6a5002c00f33a (127.15.196.194:40617)
I20260929 15:51:49.473896 21778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:58837
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:51:49.502110 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:40931
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:34283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:49.823751 22106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:49.824121 22106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:49.824316 22106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:49.831926 22106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:49.832108 22106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:49.832320 22106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:51:49.842499 22106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:34283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:40931
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:49.843945 22106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:49.845376 22106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:49.859537 22111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:49.859573 22112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:49.859622 22114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:49.860469 22106 server_base.cc:1034] running on GCE node
I20260929 15:51:49.861023 22106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:49.861889 22106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:49.863083 22106 hybrid_clock.cc:648] HybridClock initialized: now 1790697109863032 us; error 41 us; skew 500 ppm
I20260929 15:51:49.863463 22106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:49 dist-test-slave-c51v krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697109, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:49.866664 22106 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:51:49.868177 22106 webserver.cc:460] Webserver started at http://127.15.196.195:43563/ using document root <none> and password file <none>
I20260929 15:51:49.868813 22106 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:49.868911 22106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:49.869171 22106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:49.872044 22106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "235bcc5263ad440694e369b0b22825ff"
format_stamp: "Formatted at 2026-09-29 15:51:49 on dist-test-slave-c51v"
server_key: "182511980ad57622432972bf55bcb681"
server_key_iv: "a9dc728278aeefadd17f15da1bfdea9a"
server_key_version: "encryptionkey@0"
I20260929 15:51:49.872808 22106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "235bcc5263ad440694e369b0b22825ff"
format_stamp: "Formatted at 2026-09-29 15:51:49 on dist-test-slave-c51v"
server_key: "182511980ad57622432972bf55bcb681"
server_key_iv: "a9dc728278aeefadd17f15da1bfdea9a"
server_key_version: "encryptionkey@0"
I20260929 15:51:49.877877 22106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:51:49.881412 22121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:49.882850 22106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:51:49.883047 22106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "235bcc5263ad440694e369b0b22825ff"
format_stamp: "Formatted at 2026-09-29 15:51:49 on dist-test-slave-c51v"
server_key: "182511980ad57622432972bf55bcb681"
server_key_iv: "a9dc728278aeefadd17f15da1bfdea9a"
server_key_version: "encryptionkey@0"
I20260929 15:51:49.883281 22106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:49.895262 22106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:49.899817 22106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:49.900115 22106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:49.901032 22106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:49.902319 22106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:49.902422 22106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:49.902560 22106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:49.902637 22106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:49.935078 22106 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:39063
I20260929 15:51:49.935163 22233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:39063 every 8 connection(s)
I20260929 15:51:49.936471 22106 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:51:49.941700 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 22106
I20260929 15:51:49.941887 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:51:49.942639 16147 external_mini_cluster.cc:1428] Setting key 320f3bb220ff5c08690358957f969cab
Sep 29 15:51:49 dist-test-slave-c51v krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697109, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:51:49.958328 21969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:49.939053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:57295 (local address 127.15.196.254:40931)
0929 15:51:49.939301 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:49.939308 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:49.940660 (+ 1352us) server_negotiation.cc:407] Connection header received
0929 15:51:49.942583 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:49.942592 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:49.942704 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:49.942849 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:49.945102 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:49.946172 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:49.948179 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:49.948651 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:49.948846 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:49.952820 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:49.952826 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:49.952829 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:49.952862 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:49.955359 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:49.956055 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:49.956061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:49.956066 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:49.956193 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:49.956622 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:49.956633 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:49.956637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:49.956905 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:49.957065 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:49.957776 (+ 711us) server_negotiation.cc:299] Negotiation successful
0929 15:51:49.957998 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 15:51:49.959226 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:49.939682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:40931 (local address 127.15.196.195:57295)
0929 15:51:49.940494 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:49.940533 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:49.942280 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:49.943419 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:49.943429 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:49.944048 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:49.944902 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:49.944925 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:49.946357 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:49.946364 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:49.947999 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:49.948019 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:49.948911 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:49.948921 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:49.949214 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:49.949989 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:49.950017 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:49.952586 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:49.955496 (+ 2910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:49.955505 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:49.955520 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:49.955928 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:49.956305 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:49.956311 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:49.956314 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:49.956501 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:49.957034 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:49.957041 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:49.957381 (+ 340us) client_negotiation.cc:769] Sending connection context
0929 15:51:49.957658 (+ 277us) client_negotiation.cc:240] Negotiation successful
0929 15:51:49.958183 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":292,"threads_started":1}
I20260929 15:51:49.960539 22234 heartbeater.cc:347] Connected to a master server at 127.15.196.254:40931
I20260929 15:51:49.960839 22234 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:49.961513 22234 heartbeater.cc:511] Master 127.15.196.254:40931 requested a full tablet report, sending...
I20260929 15:51:49.962857 21778 ts_manager.cc:194] Registered new tserver with Master: 235bcc5263ad440694e369b0b22825ff (127.15.196.195:39063)
I20260929 15:51:49.963740 21778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:57295
I20260929 15:51:49.970137 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:51:49.980312 21967 heartbeater.cc:503] Master 127.15.196.254:40931 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/test-admin.keytab.
Sep 29 15:51:49 dist-test-slave-c51v krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:49.997726 16147 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260929 15:51:50.003021 22241 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:51:50.000478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51508 (local address 127.15.196.193:34535)
0929 15:51:50.001088 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:50.001097 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:50.001133 (+ 36us) server_negotiation.cc:407] Connection header received
0929 15:51:50.001217 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:50.001224 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:50.001448 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:50.001626 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:50.002549 (+ 923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51508: BlockingRecv error: recv got EOF from 127.0.0.1:51508 (error 108)
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":270,"threads_started":1}
W20260929 15:51:50.003129 16147 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/krb5cc' of type FILE
Sep 29 15:51:50 dist-test-slave-c51v krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:50.004889 16147 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 15:51:50 dist-test-slave-c51v krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:50.006217 16147 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 15:51:50 dist-test-slave-c51v krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:51:50.021095 22241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:50.007882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51510 (local address 127.15.196.193:34535)
0929 15:51:50.008086 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:50.008091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:50.008280 (+ 189us) server_negotiation.cc:407] Connection header received
0929 15:51:50.008427 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:50.008431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:50.008500 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:50.008583 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:50.009511 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:50.010461 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:50.011701 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:50.012289 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:50.012390 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:50.015937 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:50.015949 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:50.015964 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:50.015997 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:50.018602 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:50.019221 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:50.019227 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:50.019229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:50.019407 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:50.019715 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:50.019718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:50.019720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:50.020067 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:50.020297 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:50.020539 (+ 242us) server_negotiation.cc:299] Negotiation successful
0929 15:51:50.020803 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260929 15:51:50.026875 22241 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:51:50.025155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51524 (local address 127.15.196.193:34535)
0929 15:51:50.025362 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:50.025366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:50.025411 (+ 45us) server_negotiation.cc:407] Connection header received
0929 15:51:50.025684 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:50.025688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:50.025748 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:50.025848 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:50.026621 (+ 773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51524: BlockingRecv error: recv got EOF from 127.0.0.1:51524 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260929 15:51:50.027076 16147 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/krb5cc' of type FILE
Sep 29 15:51:50 dist-test-slave-c51v krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:50.028734 16147 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 15:51:50 dist-test-slave-c51v krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:50.029954 16147 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 15:51:50 dist-test-slave-c51v krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:51:50.044060 22241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:50.031355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51532 (local address 127.15.196.193:34535)
0929 15:51:50.031563 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:50.031569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:50.031804 (+ 235us) server_negotiation.cc:407] Connection header received
0929 15:51:50.031989 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:50.031994 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:50.032061 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:50.032160 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:50.033101 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:50.033789 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:50.035013 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:50.035529 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:50.035661 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:50.039100 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:50.039105 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:50.039108 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:50.039148 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:50.041513 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:50.042132 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:50.042144 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:50.042148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:50.042260 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:50.042631 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:50.042636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:50.042640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:50.042935 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:50.043212 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:50.043568 (+ 356us) server_negotiation.cc:299] Negotiation successful
0929 15:51:50.043786 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260929 15:51:50.048156 22241 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:51:50.046582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51544 (local address 127.15.196.193:34535)
0929 15:51:50.046766 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:50.046772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:50.046903 (+ 131us) server_negotiation.cc:407] Connection header received
0929 15:51:50.047030 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:50.047033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:50.047097 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:50.047170 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:50.047869 (+ 699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51544: BlockingRecv error: recv got EOF from 127.0.0.1:51544 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260929 15:51:50.048204 16147 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697059131395-16147-0/krb5kdc/krb5cc' of type FILE
Sep 29 15:51:50 dist-test-slave-c51v krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:50.049703 16147 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 15:51:50 dist-test-slave-c51v krb5kdc[21733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:50.050940 16147 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 15:51:50 dist-test-slave-c51v krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:51:50.064632 22241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:50.052265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51556 (local address 127.15.196.193:34535)
0929 15:51:50.052467 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:50.052475 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:50.052686 (+ 211us) server_negotiation.cc:407] Connection header received
0929 15:51:50.052870 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:50.052875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:50.052942 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:50.053044 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:50.053951 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:50.054652 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:50.055912 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:50.056326 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:50.056460 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:50.059999 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:50.060004 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:50.060007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:50.060038 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:50.062427 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:50.063032 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:50.063035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:50.063038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:50.063175 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:50.063543 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:50.063546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:50.063548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:50.063808 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:50.064038 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:50.064223 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 15:51:50.064382 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 15:51:50.066164 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 21839
I20260929 15:51:50.077668 22241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:50.066868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51558 (local address 127.15.196.193:34535)
0929 15:51:50.067049 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:50.067057 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:50.067412 (+ 355us) server_negotiation.cc:407] Connection header received
0929 15:51:50.067543 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:50.067546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:50.067598 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:50.067670 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:50.068519 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:50.069169 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:50.070288 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:50.070684 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:50.070771 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:50.072679 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:50.072684 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:50.072687 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:50.072712 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:50.075520 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:50.076059 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:50.076062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:50.076064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:50.076173 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:50.076469 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:50.076471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:50.076473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:50.076692 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:50.076883 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:50.077117 (+ 234us) server_negotiation.cc:299] Negotiation successful
0929 15:51:50.077337 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 15:51:50.078259 21962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:50.161265 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 21839
I20260929 15:51:50.178544 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 21973
Sep 29 15:51:50 dist-test-slave-c51v krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:51:50.195919 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:50.179455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.15.196.194:40617)
0929 15:51:50.180046 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:50.180054 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:50.180080 (+ 26us) server_negotiation.cc:407] Connection header received
0929 15:51:50.180149 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:50.180156 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:50.180363 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:50.180513 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:50.181612 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:50.182746 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:50.184231 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:50.185049 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:50.185175 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:50.189969 (+ 4794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:50.189984 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:50.189999 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:50.190038 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:50.193079 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:50.193781 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:50.193788 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:50.193790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:50.193910 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:50.194340 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:50.194351 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:50.194354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:50.194692 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:50.195007 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:50.195340 (+ 333us) server_negotiation.cc:299] Negotiation successful
0929 15:51:50.195618 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":307,"threads_started":1}
I20260929 15:51:50.197216 22096 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:50.279389 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 21973
I20260929 15:51:50.295964 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 22106
Sep 29 15:51:50 dist-test-slave-c51v krb5kdc[21733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:51:50.311930 22256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:50.296886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45644 (local address 127.15.196.195:39063)
0929 15:51:50.297531 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:50.297551 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:50.297582 (+ 31us) server_negotiation.cc:407] Connection header received
0929 15:51:50.297645 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:50.297652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:50.297892 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:50.298023 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:50.299084 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:50.300259 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:50.301477 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:50.302032 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:50.302128 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:50.306268 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:50.306282 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:50.306301 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:50.306349 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:50.309451 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:50.310068 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:50.310076 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:50.310080 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:50.310209 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:50.310513 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:50.310516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:50.310518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:50.310847 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:50.311085 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:50.311442 (+ 357us) server_negotiation.cc:299] Negotiation successful
0929 15:51:50.311655 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":324,"threads_started":1}
I20260929 15:51:50.313257 22229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:50.396476 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 22106
I20260929 15:51:50.412870 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 21746
I20260929 15:51:50.414193 21808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:50.516927 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 21746
2026-09-29T15:51:50Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3920 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:51:50 dist-test-slave-c51v krb5kdc[22261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:51:50 dist-test-slave-c51v krb5kdc[22261](info): set up 2 sockets
Sep 29 15:51:50 dist-test-slave-c51v krb5kdc[22261](info): commencing operation
krb5kdc: starting...
W20260929 15:51:51.618109 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:51:51 dist-test-slave-c51v krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:51:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:51:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.254.keytab.
Entry for principal oryx/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.254.keytab.
I20260929 15:51:51.831142 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:44195
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.254.keytab
--principal=oryx/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:44195
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:52.155602 22274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:52.155954 22274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:52.156126 22274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:52.163316 22274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:52.163427 22274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:52.163472 22274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:52.163602 22274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:52.163684 22274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:51:52.173345 22274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:44195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.254.keytab
--principal=oryx/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:44195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:52.174691 22274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:52.176191 22274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:52.187939 22279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:52.187866 22280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:52.187904 22282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:52.189059 22274 server_base.cc:1034] running on GCE node
I20260929 15:51:52.189877 22274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:52.191463 22274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:52.192687 22274 hybrid_clock.cc:648] HybridClock initialized: now 1790697112192626 us; error 55 us; skew 500 ppm
I20260929 15:51:52.193035 22274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:52 dist-test-slave-c51v krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:52.197332 22274 init.cc:377] Logged in from keytab as oryx/127.15.196.254@KRBTEST.COM (short username oryx)
I20260929 15:51:52.198897 22274 webserver.cc:460] Webserver started at http://127.15.196.254:35407/ using document root <none> and password file <none>
I20260929 15:51:52.199667 22274 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:52.199800 22274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:52.200114 22274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:52.203065 22274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "4a54661c127145c58a343e416a1c63da"
format_stamp: "Formatted at 2026-09-29 15:51:52 on dist-test-slave-c51v"
server_key: "3fc081f78d1193473fdb64098486ebb4"
server_key_iv: "d86fe940956377068b266cb653e2a6cd"
server_key_version: "encryptionkey@0"
I20260929 15:51:52.203899 22274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "4a54661c127145c58a343e416a1c63da"
format_stamp: "Formatted at 2026-09-29 15:51:52 on dist-test-slave-c51v"
server_key: "3fc081f78d1193473fdb64098486ebb4"
server_key_iv: "d86fe940956377068b266cb653e2a6cd"
server_key_version: "encryptionkey@0"
I20260929 15:51:52.208917 22274 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 15:51:52.212278 22289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:52.213603 22274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:51:52.213778 22274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "4a54661c127145c58a343e416a1c63da"
format_stamp: "Formatted at 2026-09-29 15:51:52 on dist-test-slave-c51v"
server_key: "3fc081f78d1193473fdb64098486ebb4"
server_key_iv: "d86fe940956377068b266cb653e2a6cd"
server_key_version: "encryptionkey@0"
I20260929 15:51:52.213972 22274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:52.234736 22274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:52.238922 22274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:52.239235 22274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:52.257934 22274 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:44195
I20260929 15:51:52.257899 22340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:44195 every 8 connection(s)
I20260929 15:51:52.259444 22274 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:51:52.263176 22341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:52.267464 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 22274
I20260929 15:51:52.267680 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:51:52.268333 16147 external_mini_cluster.cc:1428] Setting key 15eaabdda73bb96d15f14e23aeacc19e
I20260929 15:51:52.274300 22341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da: Bootstrap starting.
I20260929 15:51:52.278045 22341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:52.279338 22341 log.cc:824] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:52.282213 22341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da: No bootstrap required, opened a new log
Sep 29 15:51:52 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM
I20260929 15:51:52.290521 22341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } }
I20260929 15:51:52.290944 22341 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:52.291081 22341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a54661c127145c58a343e416a1c63da, State: Initialized, Role: FOLLOWER
I20260929 15:51:52.291610 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:52.270545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51174 (local address 127.15.196.254:44195)
0929 15:51:52.271444 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:52.271456 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:52.271513 (+ 57us) server_negotiation.cc:407] Connection header received
0929 15:51:52.273214 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:52.273238 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:52.273671 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:52.274001 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:52.275875 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:52.277197 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:52.278903 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:52.279697 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:52.279880 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:52.283899 (+ 4019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:52.283909 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:52.283927 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:52.283970 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:52.287100 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:52.287905 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:52.287913 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:52.287920 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:52.288091 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:52.288544 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:52.288548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:52.288549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:52.289134 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:52.289486 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:52.289915 (+ 429us) server_negotiation.cc:299] Negotiation successful
0929 15:51:52.290349 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":422,"threads_started":1}
I20260929 15:51:52.291836 22341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } }
I20260929 15:51:52.292129 22341 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:51:52.292282 22341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:51:52.292454 22341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:52.295120 22341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } }
I20260929 15:51:52.295632 22341 leader_election.cc:302] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a54661c127145c58a343e416a1c63da; no voters:
I20260929 15:51:52.296193 22341 leader_election.cc:288] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:51:52.296337 22346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:52.297049 22346 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [term 1 LEADER]: Becoming Leader. State: Replica: 4a54661c127145c58a343e416a1c63da, State: Running, Role: LEADER
I20260929 15:51:52.297581 22346 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } }
I20260929 15:51:52.298291 22341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:52.300227 22347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } }
I20260929 15:51:52.301102 22347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:52.300657 22348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a54661c127145c58a343e416a1c63da. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } }
I20260929 15:51:52.301288 22348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:52.302928 22351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:52.308333 22351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:52.316941 22351 catalog_manager.cc:1347] Generated new cluster ID: 894250600ca246a284b71fbfe3c54beb
I20260929 15:51:52.317054 22351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:52.331652 22351 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:51:52.333510 22351 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:52.344151 22351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da: Generated new TSK 0
I20260929 15:51:52.345163 22351 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.193.keytab.
Entry for principal oryx/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.193.keytab.
I20260929 15:51:52.403170 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44195
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.193.keytab
--principal=oryx/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:52.734041 22367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:52.734326 22367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:52.734421 22367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:52.742070 22367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:52.742219 22367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:52.742439 22367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:51:52.752141 22367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.193.keytab
--principal=oryx/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:52.753351 22367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:52.754639 22367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:52.768102 22373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:52.768103 22375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:52.768157 22372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:52.768823 22367 server_base.cc:1034] running on GCE node
I20260929 15:51:52.769477 22367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:52.770445 22367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:52.771683 22367 hybrid_clock.cc:648] HybridClock initialized: now 1790697112771633 us; error 47 us; skew 500 ppm
I20260929 15:51:52.772035 22367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:52 dist-test-slave-c51v krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:52.775449 22367 init.cc:377] Logged in from keytab as oryx/127.15.196.193@KRBTEST.COM (short username oryx)
I20260929 15:51:52.776983 22367 webserver.cc:460] Webserver started at http://127.15.196.193:46575/ using document root <none> and password file <none>
I20260929 15:51:52.777734 22367 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:52.777882 22367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:52.778196 22367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:52.781205 22367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "94640710d80f4186bd98dcc97d749008"
format_stamp: "Formatted at 2026-09-29 15:51:52 on dist-test-slave-c51v"
server_key: "ac11dcb2bfef1beb0c5da8ee3de069c4"
server_key_iv: "e70efaacb0363e09a5f65c0b706c56cc"
server_key_version: "encryptionkey@0"
I20260929 15:51:52.782075 22367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "94640710d80f4186bd98dcc97d749008"
format_stamp: "Formatted at 2026-09-29 15:51:52 on dist-test-slave-c51v"
server_key: "ac11dcb2bfef1beb0c5da8ee3de069c4"
server_key_iv: "e70efaacb0363e09a5f65c0b706c56cc"
server_key_version: "encryptionkey@0"
I20260929 15:51:52.787416 22367 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 15:51:52.790831 22382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:52.792400 22367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 15:51:52.792565 22367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "94640710d80f4186bd98dcc97d749008"
format_stamp: "Formatted at 2026-09-29 15:51:52 on dist-test-slave-c51v"
server_key: "ac11dcb2bfef1beb0c5da8ee3de069c4"
server_key_iv: "e70efaacb0363e09a5f65c0b706c56cc"
server_key_version: "encryptionkey@0"
I20260929 15:51:52.792768 22367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:52.823185 22367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:52.827791 22367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:52.828145 22367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:52.829224 22367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:52.830695 22367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:52.830783 22367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:52.830986 22367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:52.831051 22367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:52.865274 22367 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:37331
I20260929 15:51:52.865367 22494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:37331 every 8 connection(s)
I20260929 15:51:52.866608 22367 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:51:52.873761 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 22367
I20260929 15:51:52.874004 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:51:52.874807 16147 external_mini_cluster.cc:1428] Setting key 863bf69895c531c1267782c417ca43ee
Sep 29 15:51:52 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM
I20260929 15:51:52.888561 22497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:52.869276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:60905 (local address 127.15.196.254:44195)
0929 15:51:52.869729 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:52.869733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:52.870486 (+ 753us) server_negotiation.cc:407] Connection header received
0929 15:51:52.872400 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:52.872406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:52.872510 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:52.872632 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:52.874648 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:52.875517 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:52.877258 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:52.877674 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:52.877775 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:52.882003 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:52.882009 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:52.882012 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:52.882042 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:52.885172 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:52.886200 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:52.886204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:52.886206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:52.886317 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:52.886814 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:52.886817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:52.886819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:52.887070 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:52.887176 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:52.888122 (+ 946us) server_negotiation.cc:299] Negotiation successful
0929 15:51:52.888289 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":252,"threads_started":1}
I20260929 15:51:52.890179 22498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:52.869677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.193:60905)
0929 15:51:52.870323 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:52.870366 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:52.872092 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:52.872866 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:52.872875 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:52.873531 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:52.874422 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:52.874451 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:52.875670 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:52.875674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:52.877094 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:52.877111 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:52.877912 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:52.877922 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:52.878271 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:52.879051 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:52.879088 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:52.881771 (+ 2683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:52.885349 (+ 3578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:52.885362 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:52.885384 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:52.886068 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:52.886423 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:52.886430 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:52.886437 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:52.886705 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:52.887199 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:52.887210 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:52.887719 (+ 509us) client_negotiation.cc:769] Sending connection context
0929 15:51:52.888044 (+ 325us) client_negotiation.cc:240] Negotiation successful
0929 15:51:52.888655 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":254,"threads_started":1}
I20260929 15:51:52.891860 22495 heartbeater.cc:347] Connected to a master server at 127.15.196.254:44195
I20260929 15:51:52.892189 22495 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:52.892930 22495 heartbeater.cc:511] Master 127.15.196.254:44195 requested a full tablet report, sending...
I20260929 15:51:52.895147 22306 ts_manager.cc:194] Registered new tserver with Master: 94640710d80f4186bd98dcc97d749008 (127.15.196.193:37331)
I20260929 15:51:52.896718 22306 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:60905
WARNING: no policy specified for oryx/127.15.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.194.keytab.
Entry for principal oryx/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.194.keytab.
I20260929 15:51:52.924546 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44195
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.194.keytab
--principal=oryx/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:53.251783 22501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:53.252115 22501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:53.252283 22501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:53.259706 22501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:53.259864 22501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:53.260042 22501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:51:53.269788 22501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.194.keytab
--principal=oryx/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:53.271113 22501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:53.272475 22501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:53.285377 22506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:53.285430 22507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:53.285477 22509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:53.286743 22501 server_base.cc:1034] running on GCE node
I20260929 15:51:53.287458 22501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:53.288383 22501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:53.289661 22501 hybrid_clock.cc:648] HybridClock initialized: now 1790697113289567 us; error 67 us; skew 500 ppm
I20260929 15:51:53.290117 22501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:53 dist-test-slave-c51v krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697113, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:53.298702 22501 init.cc:377] Logged in from keytab as oryx/127.15.196.194@KRBTEST.COM (short username oryx)
I20260929 15:51:53.300753 22501 webserver.cc:460] Webserver started at http://127.15.196.194:36479/ using document root <none> and password file <none>
I20260929 15:51:53.301452 22501 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:53.301559 22501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:53.301823 22501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:53.305044 22501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "5cf3ae47c7cb4f09a336e751fa90261f"
format_stamp: "Formatted at 2026-09-29 15:51:53 on dist-test-slave-c51v"
server_key: "02d0eb95d251e430f8135b9acaea0e69"
server_key_iv: "848c947f992e65ee85c132c6d4a25cb7"
server_key_version: "encryptionkey@0"
I20260929 15:51:53.305784 22501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "5cf3ae47c7cb4f09a336e751fa90261f"
format_stamp: "Formatted at 2026-09-29 15:51:53 on dist-test-slave-c51v"
server_key: "02d0eb95d251e430f8135b9acaea0e69"
server_key_iv: "848c947f992e65ee85c132c6d4a25cb7"
server_key_version: "encryptionkey@0"
I20260929 15:51:53.311172 22501 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 15:51:53.314610 22516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:53.316063 22501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:51:53.316238 22501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "5cf3ae47c7cb4f09a336e751fa90261f"
format_stamp: "Formatted at 2026-09-29 15:51:53 on dist-test-slave-c51v"
server_key: "02d0eb95d251e430f8135b9acaea0e69"
server_key_iv: "848c947f992e65ee85c132c6d4a25cb7"
server_key_version: "encryptionkey@0"
I20260929 15:51:53.316429 22501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:53.335986 22501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:53.340344 22501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:53.340674 22501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:53.341646 22501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:53.343015 22501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:53.343086 22501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:53.343236 22501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:53.343344 22501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260929 15:51:53.376390 22501 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:33961
I20260929 15:51:53.376430 22628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:33961 every 8 connection(s)
I20260929 15:51:53.377723 22501 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:51:53.384536 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 22501
I20260929 15:51:53.384727 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:51:53.385179 16147 external_mini_cluster.cc:1428] Setting key 28fac1bff87bce1ad23971b0e0c02443
Sep 29 15:51:53 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697113, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM
I20260929 15:51:53.399045 22497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:53.380274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:49535 (local address 127.15.196.254:44195)
0929 15:51:53.380493 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:53.380507 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:53.381669 (+ 1162us) server_negotiation.cc:407] Connection header received
0929 15:51:53.383392 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:53.383397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:53.383467 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:53.383560 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:53.385282 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:53.386159 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:53.387796 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:53.388214 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:53.388355 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:53.392329 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:53.392334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:53.392337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:53.392368 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:53.395845 (+ 3477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:53.396640 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:53.396648 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:53.396655 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:53.396823 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:53.397236 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:53.397244 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:53.397250 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:53.397646 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:53.397783 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:53.398448 (+ 665us) server_negotiation.cc:299] Negotiation successful
0929 15:51:53.398679 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260929 15:51:53.400197 22631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:53.380809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.194:49535)
0929 15:51:53.381506 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:53.381543 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:53.383099 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:53.383770 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:53.383777 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:53.384445 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:53.385106 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:53.385123 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:53.386325 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:53.386329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:53.387651 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:53.387663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:53.388449 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:53.388456 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:53.388695 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:53.389386 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:53.389412 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:53.392095 (+ 2683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:53.396057 (+ 3962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:53.396075 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:53.396089 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:53.396531 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:53.396929 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:53.396932 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:53.396934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:53.397138 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:53.397765 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:53.397771 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:53.398094 (+ 323us) client_negotiation.cc:769] Sending connection context
0929 15:51:53.398366 (+ 272us) client_negotiation.cc:240] Negotiation successful
0929 15:51:53.398959 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":266,"threads_started":1}
I20260929 15:51:53.401750 22629 heartbeater.cc:347] Connected to a master server at 127.15.196.254:44195
I20260929 15:51:53.402060 22629 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:53.403038 22629 heartbeater.cc:511] Master 127.15.196.254:44195 requested a full tablet report, sending...
I20260929 15:51:53.404790 22306 ts_manager.cc:194] Registered new tserver with Master: 5cf3ae47c7cb4f09a336e751fa90261f (127.15.196.194:33961)
I20260929 15:51:53.406049 22306 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:49535
WARNING: no policy specified for oryx/127.15.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.195.keytab.
Entry for principal oryx/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.195.keytab.
I20260929 15:51:53.432543 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44195
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.195.keytab
--principal=oryx/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:53.756525 22634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:53.756798 22634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:53.756899 22634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:53.764120 22634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:53.764226 22634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:53.764338 22634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:51:53.773962 22634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.195.keytab
--principal=oryx/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:53.775228 22634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:53.776535 22634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:53.790270 22639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:53.790190 22642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:53.790189 22640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:53.790699 22634 server_base.cc:1034] running on GCE node
I20260929 15:51:53.791697 22634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:53.792676 22634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:53.793957 22634 hybrid_clock.cc:648] HybridClock initialized: now 1790697113793901 us; error 44 us; skew 500 ppm
I20260929 15:51:53.794339 22634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:53 dist-test-slave-c51v krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697113, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:53.797690 22634 init.cc:377] Logged in from keytab as oryx/127.15.196.195@KRBTEST.COM (short username oryx)
I20260929 15:51:53.799234 22634 webserver.cc:460] Webserver started at http://127.15.196.195:42519/ using document root <none> and password file <none>
I20260929 15:51:53.800086 22634 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:53.800232 22634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:53.800539 22634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:53.803638 22634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "d2e4767290964f7fa24e766c3c1248b7"
format_stamp: "Formatted at 2026-09-29 15:51:53 on dist-test-slave-c51v"
server_key: "e087f5a0d4d59e4e96ddba31aa3b2afe"
server_key_iv: "55eba561429b2fe3b7d48e03770bad01"
server_key_version: "encryptionkey@0"
I20260929 15:51:53.804462 22634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "d2e4767290964f7fa24e766c3c1248b7"
format_stamp: "Formatted at 2026-09-29 15:51:53 on dist-test-slave-c51v"
server_key: "e087f5a0d4d59e4e96ddba31aa3b2afe"
server_key_iv: "55eba561429b2fe3b7d48e03770bad01"
server_key_version: "encryptionkey@0"
I20260929 15:51:53.810021 22634 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 15:51:53.813580 22649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:53.814970 22634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:51:53.815155 22634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "d2e4767290964f7fa24e766c3c1248b7"
format_stamp: "Formatted at 2026-09-29 15:51:53 on dist-test-slave-c51v"
server_key: "e087f5a0d4d59e4e96ddba31aa3b2afe"
server_key_iv: "55eba561429b2fe3b7d48e03770bad01"
server_key_version: "encryptionkey@0"
I20260929 15:51:53.815403 22634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:53.833550 22634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:53.837924 22634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:53.838246 22634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:53.839229 22634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:53.840601 22634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:53.840710 22634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:53.840864 22634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:53.840936 22634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:53.876858 22634 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:33907
I20260929 15:51:53.876893 22761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:33907 every 8 connection(s)
I20260929 15:51:53.878165 22634 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:51:53.882879 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 22634
I20260929 15:51:53.883059 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:51:53.883632 16147 external_mini_cluster.cc:1428] Setting key caaddf8afeffb464bcf7901b801100d4
Sep 29 15:51:53 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697113, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM
I20260929 15:51:53.898280 22497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:53.880811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:48697 (local address 127.15.196.254:44195)
0929 15:51:53.880990 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:53.880995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:53.882162 (+ 1167us) server_negotiation.cc:407] Connection header received
0929 15:51:53.884157 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:53.884162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:53.884262 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:53.884380 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:53.885855 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:53.886627 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:53.888544 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:53.889012 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:53.889166 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:53.892759 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:53.892766 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:53.892771 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:53.892816 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:53.895095 (+ 2279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:53.895884 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:53.895889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:53.895893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:53.896030 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:53.896471 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:53.896474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:53.896476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:53.896736 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:53.896883 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:53.897674 (+ 791us) server_negotiation.cc:299] Negotiation successful
0929 15:51:53.897928 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 15:51:53.899468 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:53.881310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.195:48697)
0929 15:51:53.881983 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:53.882034 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:53.883868 (+ 1834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:53.884619 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:53.884628 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:53.885142 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:53.885686 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:53.885702 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:53.886768 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:53.886774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:53.888370 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:53.888384 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:53.889254 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:53.889264 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:53.889543 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:53.890177 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:53.890206 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:53.892554 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:53.895243 (+ 2689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:53.895252 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:53.895269 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:53.895756 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:53.896137 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:53.896143 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:53.896146 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:53.896328 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:53.896870 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:53.896878 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:53.897287 (+ 409us) client_negotiation.cc:769] Sending connection context
0929 15:51:53.897545 (+ 258us) client_negotiation.cc:240] Negotiation successful
0929 15:51:53.898089 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":255,"threads_started":1}
I20260929 15:51:53.900259 22495 heartbeater.cc:503] Master 127.15.196.254:44195 was elected leader, sending a full tablet report...
I20260929 15:51:53.901068 22762 heartbeater.cc:347] Connected to a master server at 127.15.196.254:44195
I20260929 15:51:53.901372 22762 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:53.902173 22762 heartbeater.cc:511] Master 127.15.196.254:44195 requested a full tablet report, sending...
I20260929 15:51:53.903503 22306 ts_manager.cc:194] Registered new tserver with Master: d2e4767290964f7fa24e766c3c1248b7 (127.15.196.195:33907)
I20260929 15:51:53.904261 22306 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:48697
I20260929 15:51:53.911103 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.196.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.196.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.196.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.253.keytab.
Entry for principal oryx/127.15.196.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.253.keytab.
I20260929 15:51:53.958927 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.253:45983
--webserver_interface=127.15.196.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.253.keytab
--principal=oryx/127.15.196.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:44195,127.15.196.253:45983
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:54.291399 22767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:54.291699 22767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:54.291862 22767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:54.299571 22767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:54.299705 22767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:54.299783 22767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:54.299866 22767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:54.299924 22767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:51:54.310109 22767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:44195,127.15.196.253:45983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.253.keytab
--principal=oryx/127.15.196.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.253:45983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.196.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
W20260929 15:51:54.310510 22767 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.196.254:44195,127.15.196.253:45983'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260929 15:51:54.311618 22767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:54.312965 22767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:54.325085 22772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:54.325205 22773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:54.325174 22775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:54.325905 22767 server_base.cc:1034] running on GCE node
I20260929 15:51:54.326606 22767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:54.327982 22767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:54.329231 22767 hybrid_clock.cc:648] HybridClock initialized: now 1790697114329146 us; error 51 us; skew 500 ppm
I20260929 15:51:54.329563 22767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:54 dist-test-slave-c51v krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697114, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:54.332623 22767 init.cc:377] Logged in from keytab as oryx/127.15.196.253@KRBTEST.COM (short username oryx)
I20260929 15:51:54.334043 22767 webserver.cc:460] Webserver started at http://127.15.196.253:33951/ using document root <none> and password file <none>
I20260929 15:51:54.334728 22767 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:54.334857 22767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:54.335186 22767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:54.338241 22767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/data/instance:
uuid: "5f752fb87cd64afc99afd30cd9da9cee"
format_stamp: "Formatted at 2026-09-29 15:51:54 on dist-test-slave-c51v"
server_key: "64556b84375da3614ac32691e1720fe4"
server_key_iv: "235cfbc58df415c99cd3fcae7b2c1470"
server_key_version: "encryptionkey@0"
I20260929 15:51:54.339053 22767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/wal/instance:
uuid: "5f752fb87cd64afc99afd30cd9da9cee"
format_stamp: "Formatted at 2026-09-29 15:51:54 on dist-test-slave-c51v"
server_key: "64556b84375da3614ac32691e1720fe4"
server_key_iv: "235cfbc58df415c99cd3fcae7b2c1470"
server_key_version: "encryptionkey@0"
I20260929 15:51:54.344131 22767 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 15:51:54.347506 22782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:54.348815 22767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:51:54.348987 22767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/wal
uuid: "5f752fb87cd64afc99afd30cd9da9cee"
format_stamp: "Formatted at 2026-09-29 15:51:54 on dist-test-slave-c51v"
server_key: "64556b84375da3614ac32691e1720fe4"
server_key_iv: "235cfbc58df415c99cd3fcae7b2c1470"
server_key_version: "encryptionkey@0"
I20260929 15:51:54.349188 22767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:54.377140 22767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:54.381399 22767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:54.381678 22767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:54.400887 22767 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.253:45983
I20260929 15:51:54.400905 22833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.253:45983 every 8 connection(s)
I20260929 15:51:54.402285 22767 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/data/info.pb
I20260929 15:51:54.406194 22834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:54.408679 22629 heartbeater.cc:503] Master 127.15.196.254:44195 was elected leader, sending a full tablet report...
I20260929 15:51:54.409723 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 22767
I20260929 15:51:54.414547 22834 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:51:54 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.196.253@KRBTEST.COM
Sep 29 15:51:54 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697114, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.253@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM
I20260929 15:51:54.433316 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:54.417792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51176 (local address 127.15.196.254:44195)
0929 15:51:54.418356 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:54.418364 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:54.418871 (+ 507us) server_negotiation.cc:407] Connection header received
0929 15:51:54.418990 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:54.418995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:54.419067 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:54.419183 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:54.420446 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.421305 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.422683 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.423079 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.423208 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:54.426831 (+ 3623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:54.426839 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:54.426845 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:54.426890 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:54.430167 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:54.430986 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:54.430990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:54.430992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:54.431105 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:54.431554 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:54.431558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:54.431561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:54.431927 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:54.432085 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:54.432761 (+ 676us) server_negotiation.cc:299] Negotiation successful
0929 15:51:54.432987 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":342,"threads_started":1}
I20260929 15:51:54.434327 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:54.412165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48488 (local address 127.15.196.253:45983)
0929 15:51:54.412915 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:54.412930 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:54.412986 (+ 56us) server_negotiation.cc:407] Connection header received
0929 15:51:54.415100 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:54.415131 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:54.415695 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:54.416164 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:54.417605 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.419707 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.421065 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.421823 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.422026 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:54.426379 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:54.426387 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:54.426398 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:54.426426 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:54.429850 (+ 3424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:54.430610 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:54.430617 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:54.430619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:54.430768 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:54.431198 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:54.431202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:54.431205 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:54.431927 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:54.432225 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:54.432628 (+ 403us) server_negotiation.cc:299] Negotiation successful
0929 15:51:54.433299 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":236,"server-negotiator.queue_time_us":521,"thread_start_us":315,"threads_started":1}
I20260929 15:51:54.434401 22839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:54.417927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.0.0.1:51176)
0929 15:51:54.418682 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:54.418716 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:54.418879 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:54.419273 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:54.419314 (+ 41us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:54.420048 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:54.420308 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:54.420316 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.421431 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.421438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:54.422534 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:54.422546 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.423321 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.423329 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:54.423478 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:54.424086 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:54.424106 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:54.426640 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:54.430308 (+ 3668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:54.430322 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:54.430337 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:54.430876 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:54.431243 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:54.431246 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:54.431249 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:54.431436 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:54.432081 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:54.432089 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:54.432339 (+ 250us) client_negotiation.cc:769] Sending connection context
0929 15:51:54.432627 (+ 288us) client_negotiation.cc:240] Negotiation successful
0929 15:51:54.433175 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":4,"thread_start_us":404,"threads_started":1}
I20260929 15:51:54.436305 22834 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:51:54 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697114, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.253@KRBTEST.COM for oryx/127.15.196.253@KRBTEST.COM
I20260929 15:51:54.448058 22839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:54.437071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:45983 (local address 127.0.0.1:48504)
0929 15:51:54.437327 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:54.437341 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:54.437462 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:54.437755 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:54.437759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:54.438092 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:54.438212 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:54.438220 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.439151 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.439157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:54.440089 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:54.440100 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.440742 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.440748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:54.440932 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:54.441411 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:54.441427 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:54.443460 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:54.445952 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:54.445979 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:54.445982 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:54.446320 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:54.446667 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:54.446670 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:54.446673 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:54.446807 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:54.447252 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:54.447255 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:54.447440 (+ 185us) client_negotiation.cc:769] Sending connection context
0929 15:51:54.447552 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 15:51:54.447754 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260929 15:51:54.448171 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:54.437056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.15.196.253:45983)
0929 15:51:54.437222 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:54.437227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:54.437402 (+ 175us) server_negotiation.cc:407] Connection header received
0929 15:51:54.437583 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:54.437588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:54.437648 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:54.437739 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:54.438336 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.439018 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.440226 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.440607 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.440738 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:54.443632 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:54.443637 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:54.443641 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:54.443666 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:54.445831 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:54.446415 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:54.446418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:54.446420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:54.446545 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:54.446900 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:54.446903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:54.446905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:54.447156 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:54.447259 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:54.447629 (+ 370us) server_negotiation.cc:299] Negotiation successful
0929 15:51:54.447869 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":56}
I20260929 15:51:54.452793 22834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: Bootstrap starting.
I20260929 15:51:54.455881 22834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:54.456840 22834 log.cc:824] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:54.459481 22834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: No bootstrap required, opened a new log
I20260929 15:51:54.466526 22834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } }
I20260929 15:51:54.466837 22834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:54.466898 22834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f752fb87cd64afc99afd30cd9da9cee, State: Initialized, Role: FOLLOWER
I20260929 15:51:54.467482 22834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } }
I20260929 15:51:54.468299 22841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } } }
I20260929 15:51:54.468624 22841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:51:54.469237 22834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: configured and running, proceeding with master startup.
W20260929 15:51:54.477485 22852 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 15:51:54.477655 22852 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 15:51:54.480171 22767 master_runner.cc:428] Detected that this master 5f752fb87cd64afc99afd30cd9da9cee is joining an existing cluster
I20260929 15:51:54.480260 22767 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.196.253:45983
I20260929 15:51:54.502561 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:54.484544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48518 (local address 127.15.196.253:45983)
0929 15:51:54.484788 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:54.484800 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:54.486376 (+ 1576us) server_negotiation.cc:407] Connection header received
0929 15:51:54.486552 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:54.486559 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:54.486622 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:54.486714 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:54.488340 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.489473 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.491796 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.492523 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.492636 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:54.495575 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:54.495582 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:54.495592 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:54.495625 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:54.499423 (+ 3798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:54.500331 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:54.500343 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:54.500347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:54.500489 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:54.500913 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:54.500917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:54.500920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:54.501314 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:54.501465 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:54.501952 (+ 487us) server_negotiation.cc:299] Negotiation successful
0929 15:51:54.502183 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 15:51:54.503011 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:54.485139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51180 (local address 127.15.196.254:44195)
0929 15:51:54.485377 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:54.485385 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:54.486079 (+ 694us) server_negotiation.cc:407] Connection header received
0929 15:51:54.486350 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:54.486356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:54.486475 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:54.486591 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:54.488364 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.489387 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.492005 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.492795 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.492908 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:54.495607 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:54.495615 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:54.495622 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:54.495669 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:54.499583 (+ 3914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:54.500507 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:54.500514 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:54.500520 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:54.500666 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:54.501132 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:54.501140 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:54.501145 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:54.501524 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:54.501654 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:54.502305 (+ 651us) server_negotiation.cc:299] Negotiation successful
0929 15:51:54.502565 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 15:51:54.504187 22858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:54.484701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:45983 (local address 127.0.0.1:48518)
0929 15:51:54.486294 (+ 1593us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:54.486311 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:54.486436 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:54.486758 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:54.486765 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:54.487598 (+ 833us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:54.488191 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:54.488205 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.489748 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.489756 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:54.491641 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:54.491655 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.492941 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.492947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:54.493107 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:54.493785 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:54.493812 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:54.495385 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:54.499616 (+ 4231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:54.499621 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:54.499626 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:54.500213 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:54.500626 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:54.500631 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:54.500634 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:54.500798 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:54.501541 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:54.501545 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:54.501695 (+ 150us) client_negotiation.cc:769] Sending connection context
0929 15:51:54.503335 (+ 1640us) client_negotiation.cc:240] Negotiation successful
0929 15:51:54.503707 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1459,"mutex_wait_us":165,"thread_start_us":883,"threads_started":1}
I20260929 15:51:54.504611 22859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:54.484581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.0.0.1:51180)
0929 15:51:54.485964 (+ 1383us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:54.485988 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:54.486205 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:54.486782 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:54.486787 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:54.487573 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:54.488193 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:54.488204 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.489595 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.489601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:54.491819 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:54.491836 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.493210 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.493219 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:54.493437 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:54.493965 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:54.493999 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:54.495386 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:54.499752 (+ 4366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:54.499759 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:54.499766 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:54.500370 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:54.500805 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:54.500812 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:54.500819 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:54.500998 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:54.501736 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:54.501743 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:54.502030 (+ 287us) client_negotiation.cc:769] Sending connection context
0929 15:51:54.503264 (+ 1234us) client_negotiation.cc:240] Negotiation successful
0929 15:51:54.503952 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1191,"mutex_wait_us":386,"spinlock_wait_cycles":27008,"thread_start_us":1400,"threads_started":1}
WARNING: no policy specified for oryx/127.15.196.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.196.252@KRBTEST.COM" created.
Sep 29 15:51:54 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.254@KRBTEST.COM for oryx/127.15.196.253@KRBTEST.COM
I20260929 15:51:54.528755 22861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:54.511250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:45983 (local address 127.0.0.1:48526)
0929 15:51:54.511951 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:54.512005 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:54.512235 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:54.512770 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:54.512779 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:54.513681 (+ 902us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:54.513985 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:54.513995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.515425 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.515429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:54.516570 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:54.516583 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.517592 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.517598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:54.517794 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:54.518585 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:54.518614 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:54.521611 (+ 2997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:54.525313 (+ 3702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:54.525328 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:54.525339 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:54.525985 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:54.526448 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:54.526456 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:54.526462 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:54.526656 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:54.527432 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:54.527442 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:54.527679 (+ 237us) client_negotiation.cc:769] Sending connection context
0929 15:51:54.527920 (+ 241us) client_negotiation.cc:240] Negotiation successful
0929 15:51:54.528306 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":305,"threads_started":1}
I20260929 15:51:54.528769 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:54.511305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48526 (local address 127.15.196.253:45983)
0929 15:51:54.511568 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:54.511575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:54.512091 (+ 516us) server_negotiation.cc:407] Connection header received
0929 15:51:54.512464 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:54.512470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:54.512590 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:54.512706 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:54.514133 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.515262 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.516798 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.517404 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.517574 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:54.521797 (+ 4223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:54.521803 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:54.521808 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:54.521850 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:54.525122 (+ 3272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:54.526142 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:54.526150 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:54.526157 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:54.526314 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:54.526784 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:54.526793 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:54.526799 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:54.527205 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:54.527374 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:54.528051 (+ 677us) server_negotiation.cc:299] Negotiation successful
0929 15:51:54.528326 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260929 15:51:54.531067 22306 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.196.253:45983: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee"
member_type: NON_VOTER
last_known_addr {
host: "127.15.196.253"
port: 45983
}
attrs {
promote: true
}
}
dest_uuid: "4a54661c127145c58a343e416a1c63da"
cas_config_opid_index: -1
I20260929 15:51:54.532367 22306 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: NON_VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: true } }
I20260929 15:51:54.542660 22864 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5f752fb87cd64afc99afd30cd9da9cee (127.15.196.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: NON_VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: true } } }
I20260929 15:51:54.543565 22809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:54.547420 22809 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 FOLLOWER]: Refusing update from remote peer 4a54661c127145c58a343e416a1c63da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260929 15:51:54.547447 22864 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.196.253:45983
I20260929 15:51:54.548178 22767 master.cc:537] Master@127.15.196.253:45983 shutting down...
I20260929 15:51:54.548888 22864 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: NON_VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:54.548795 22865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: NON_VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: true } } }
I20260929 15:51:54.549436 22865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.196.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.252.keytab.
Entry for principal oryx/127.15.196.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.252.keytab.
I20260929 15:51:54.551848 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.252:34407
--webserver_interface=127.15.196.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.252.keytab
--principal=oryx/127.15.196.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:44195,127.15.196.253:45983,127.15.196.252:34407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:51:54.558323 22290 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260929 15:51:54.560035 22290 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da -> Peer 5f752fb87cd64afc99afd30cd9da9cee (127.15.196.253:45983): Couldn't send request to peer 5f752fb87cd64afc99afd30cd9da9cee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 15:51:54.561255 22767 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:54.561657 22767 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:54.561851 22767 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: stopping tablet replica
I20260929 15:51:54.567896 22767 master.cc:552] Master@127.15.196.253:45983 shutdown complete.
I20260929 15:51:54.567987 22767 master_runner.cc:305] Clearing existing system tablet
I20260929 15:51:54.568344 22767 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:54.568449 22767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:54.571266 22767 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260929 15:51:54.572889 22870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:54.573203 22767 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:54.573488 22767 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 15:51:54.573572 22767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/wal
uuid: "5f752fb87cd64afc99afd30cd9da9cee"
format_stamp: "Formatted at 2026-09-29 15:51:54 on dist-test-slave-c51v"
server_key: "64556b84375da3614ac32691e1720fe4"
server_key_iv: "235cfbc58df415c99cd3fcae7b2c1470"
server_key_version: "encryptionkey@0"
I20260929 15:51:54.576952 22767 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:54.582201 22767 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 15:51:54.582455 22767 log.cc:1196] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260929 15:51:54.582867 22767 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: Deleting consensus metadata
I20260929 15:51:54.583118 22767 master_runner.cc:315] Copying system tablet from 127.15.196.254:44195
I20260929 15:51:54.585650 22767 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: tablet copy: Beginning tablet copy session from remote peer at address 127.15.196.254:44195
I20260929 15:51:54.598147 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:54.586407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51192 (local address 127.15.196.254:44195)
0929 15:51:54.586638 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:54.586648 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:54.587111 (+ 463us) server_negotiation.cc:407] Connection header received
0929 15:51:54.587367 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:54.587372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:54.587461 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:54.587574 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:54.588342 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.589069 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.590372 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.590793 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.590926 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:54.592680 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:54.592686 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:54.592691 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:54.592731 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:54.595386 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:54.596132 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:54.596139 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:54.596143 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:54.596272 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:54.596682 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:54.596687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:54.596690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:54.596960 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:54.597173 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:54.597521 (+ 348us) server_negotiation.cc:299] Negotiation successful
0929 15:51:54.597729 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 15:51:54.598169 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:54.586371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.0.0.1:51192)
0929 15:51:54.586991 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:54.587013 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:54.587178 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:54.587599 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:54.587604 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:54.587990 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:54.588198 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:54.588207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.589209 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.589214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:54.590222 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:54.590233 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:54.590938 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:54.590944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:54.591119 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:54.591802 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:54.591820 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:54.592494 (+ 674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:54.595618 (+ 3124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:54.595625 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:54.595629 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:54.596036 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:54.596382 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:54.596387 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:54.596390 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:54.596531 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:54.597086 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:54.597090 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:54.597278 (+ 188us) client_negotiation.cc:769] Sending connection context
0929 15:51:54.597421 (+ 143us) client_negotiation.cc:240] Negotiation successful
0929 15:51:54.597730 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":410,"threads_started":1}
I20260929 15:51:54.599360 22326 tablet_copy_service.cc:140] P 4a54661c127145c58a343e416a1c63da: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5f752fb87cd64afc99afd30cd9da9cee ({username='oryx', principal='oryx/127.15.196.253@KRBTEST.COM'} at 127.0.0.1:51192)
I20260929 15:51:54.599649 22326 tablet_copy_service.cc:161] P 4a54661c127145c58a343e416a1c63da: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5f752fb87cd64afc99afd30cd9da9cee at {username='oryx', principal='oryx/127.15.196.253@KRBTEST.COM'} at 127.0.0.1:51192: session id = 5f752fb87cd64afc99afd30cd9da9cee-00000000000000000000000000000000
I20260929 15:51:54.602779 22326 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 15:51:54.604823 22767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 15:51:54.610028 22767 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: tablet copy: Starting download of 0 data blocks...
I20260929 15:51:54.610320 22767 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: tablet copy: Starting download of 1 WAL segments...
I20260929 15:51:54.612635 22767 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 15:51:54.615710 22326 tablet_copy_service.cc:342] P 4a54661c127145c58a343e416a1c63da: Request end of tablet copy session 5f752fb87cd64afc99afd30cd9da9cee-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.196.253@KRBTEST.COM'} at 127.0.0.1:51192
I20260929 15:51:54.615902 22326 tablet_copy_service.cc:433] P 4a54661c127145c58a343e416a1c63da: ending tablet copy session 5f752fb87cd64afc99afd30cd9da9cee-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5f752fb87cd64afc99afd30cd9da9cee
I20260929 15:51:54.626906 22767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:54.629606 22877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:54.633010 22878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:54.643597 22767 server_base.cc:1034] running on GCE node
W20260929 15:51:54.643749 22880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:54.644248 22767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:54.644920 22767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:54.646159 22767 hybrid_clock.cc:648] HybridClock initialized: now 1790697114646082 us; error 64 us; skew 500 ppm
I20260929 15:51:54.646394 22767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:54 dist-test-slave-c51v krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697114, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:54.648326 22767 init.cc:377] Logged in from keytab as oryx/127.15.196.253@KRBTEST.COM (short username oryx)
I20260929 15:51:54.649564 22767 webserver.cc:460] Webserver started at http://127.15.196.253:37547/ using document root <none> and password file <none>
I20260929 15:51:54.649899 22767 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:54.649998 22767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:54.652486 22767 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260929 15:51:54.654119 22887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:54.654546 22767 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260929 15:51:54.654695 22767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/wal
uuid: "5f752fb87cd64afc99afd30cd9da9cee"
format_stamp: "Formatted at 2026-09-29 15:51:54 on dist-test-slave-c51v"
server_key: "64556b84375da3614ac32691e1720fe4"
server_key_iv: "235cfbc58df415c99cd3fcae7b2c1470"
server_key_version: "encryptionkey@0"
I20260929 15:51:54.654843 22767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:54.669423 22767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:54.672868 22767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:54.695452 22767 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.253:45983
I20260929 15:51:54.695482 22938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.253:45983 every 8 connection(s)
I20260929 15:51:54.696460 22767 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-1/data/info.pb
I20260929 15:51:54.703785 22939 sys_catalog.cc:263] Verifying existing consensus state
I20260929 15:51:54.709136 22939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: Bootstrap starting.
I20260929 15:51:54.723905 22939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 15:51:54.724375 22939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: Bootstrap complete.
I20260929 15:51:54.725214 22939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: NON_VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: true } }
I20260929 15:51:54.725443 22939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f752fb87cd64afc99afd30cd9da9cee, State: Initialized, Role: LEARNER
I20260929 15:51:54.725709 22939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: NON_VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: true } }
I20260929 15:51:54.726505 22942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: NON_VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: true } } }
I20260929 15:51:54.726848 22939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:54.726887 22942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: This master's current role is: LEARNER
I20260929 15:51:54.735216 22953 catalog_manager.cc:1259] Loaded cluster ID: 894250600ca246a284b71fbfe3c54beb
I20260929 15:51:54.735358 22953 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: loading cluster ID for follower catalog manager: success
I20260929 15:51:54.737888 22953 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: acquiring CA information for follower catalog manager: success
I20260929 15:51:54.739722 22953 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 15:51:54.906723 22762 heartbeater.cc:503] Master 127.15.196.254:44195 was elected leader, sending a full tablet report...
W20260929 15:51:54.975973 22867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:54.976339 22867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:54.976536 22867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:54.983976 22867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:51:54.984086 22867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:54.984169 22867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:54.984268 22867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:51:54.984330 22867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:51:54.994361 22867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:44195,127.15.196.253:45983,127.15.196.252:34407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.252.keytab
--principal=oryx/127.15.196.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.252:34407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.196.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:54.995788 22867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:54.997362 22867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:55.009202 22961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:55.009270 22959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:55.009147 22958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:55.012948 22867 server_base.cc:1034] running on GCE node
I20260929 15:51:55.013682 22867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:55.015020 22867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:55.016247 22867 hybrid_clock.cc:648] HybridClock initialized: now 1790697115016199 us; error 66 us; skew 500 ppm
I20260929 15:51:55.016594 22867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:55 dist-test-slave-c51v krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697115, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:55.020027 22867 init.cc:377] Logged in from keytab as oryx/127.15.196.252@KRBTEST.COM (short username oryx)
I20260929 15:51:55.021677 22867 webserver.cc:460] Webserver started at http://127.15.196.252:34285/ using document root <none> and password file <none>
I20260929 15:51:55.022380 22867 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:55.022485 22867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:55.022729 22867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:51:55.025997 22867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/data/instance:
uuid: "1cb3483982a74bf5b724b88fee7f15e3"
format_stamp: "Formatted at 2026-09-29 15:51:55 on dist-test-slave-c51v"
server_key: "c1eee3d85e582cf35c86edde80b4804e"
server_key_iv: "429b04da8f9dd289d592ebf04f693ebf"
server_key_version: "encryptionkey@0"
I20260929 15:51:55.027061 22867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/wal/instance:
uuid: "1cb3483982a74bf5b724b88fee7f15e3"
format_stamp: "Formatted at 2026-09-29 15:51:55 on dist-test-slave-c51v"
server_key: "c1eee3d85e582cf35c86edde80b4804e"
server_key_iv: "429b04da8f9dd289d592ebf04f693ebf"
server_key_version: "encryptionkey@0"
I20260929 15:51:55.032899 22867 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 15:51:55.036303 22968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:55.037731 22867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:51:55.037864 22867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/wal
uuid: "1cb3483982a74bf5b724b88fee7f15e3"
format_stamp: "Formatted at 2026-09-29 15:51:55 on dist-test-slave-c51v"
server_key: "c1eee3d85e582cf35c86edde80b4804e"
server_key_iv: "429b04da8f9dd289d592ebf04f693ebf"
server_key_version: "encryptionkey@0"
I20260929 15:51:55.038022 22867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:55.056447 22867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:55.060544 22867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:55.060774 22867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:55.073387 22979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.065708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48528 (local address 127.15.196.253:45983)
0929 15:51:55.066780 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:55.066788 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:55.066815 (+ 27us) server_negotiation.cc:407] Connection header received
0929 15:51:55.066908 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:55.066917 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:55.067016 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:55.067152 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:55.067658 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.069046 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.070871 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.072115 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.072314 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.072552 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:55.073053 (+ 501us) server_negotiation.cc:299] Negotiation successful
0929 15:51:55.073153 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":896,"thread_start_us":695,"threads_started":1}
I20260929 15:51:55.073408 22977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.065442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:45983 (local address 127.0.0.1:48528)
0929 15:51:55.065975 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:55.065999 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:55.066136 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:55.067248 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:55.067257 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:55.067329 (+ 72us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:55.067510 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.067518 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.069186 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.069191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.070727 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.070741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.072325 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.072331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.072587 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.072662 (+ 75us) client_negotiation.cc:769] Sending connection context
0929 15:51:55.072775 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 15:51:55.072950 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":237,"threads_started":1}
I20260929 15:51:55.074869 22914 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260929 15:51:55.075670 22942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a54661c127145c58a343e416a1c63da. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: NON_VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: true } } }
I20260929 15:51:55.075932 22942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: This master's current role is: LEARNER
I20260929 15:51:55.081346 22867 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.252:34407
I20260929 15:51:55.081377 23022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.252:34407 every 8 connection(s)
I20260929 15:51:55.082767 22867 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/data/info.pb
I20260929 15:51:55.086481 23023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:55.088402 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 22867
I20260929 15:51:55.095570 23023 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:51:55 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.196.252@KRBTEST.COM
Sep 29 15:51:55 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697115, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.252@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM
I20260929 15:51:55.115044 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.090517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41768 (local address 127.15.196.252:34407)
0929 15:51:55.091383 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:55.091396 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:55.091444 (+ 48us) server_negotiation.cc:407] Connection header received
0929 15:51:55.092960 (+ 1516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:55.092992 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:55.093491 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:55.093856 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:55.095375 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.096734 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.098281 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.099509 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.099710 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.106023 (+ 6313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:55.106034 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:55.106048 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:55.106091 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:55.109651 (+ 3560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.110522 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.110538 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.110547 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.110736 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.111217 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.111223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.111226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.111976 (+ 750us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:55.112274 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:55.113066 (+ 792us) server_negotiation.cc:299] Negotiation successful
0929 15:51:55.113522 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":374,"threads_started":1}
I20260929 15:51:55.119833 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.098454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.0.0.1:51194)
0929 15:51:55.100743 (+ 2289us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:55.100793 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:55.101032 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:55.101589 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:55.101617 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:55.102607 (+ 990us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:55.102993 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.103009 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.104498 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.104504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.106350 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.106363 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.107628 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.107634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.107838 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.109882 (+ 2044us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:55.109909 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:55.112704 (+ 2795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:55.116553 (+ 3849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.116567 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.116575 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.117264 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.117725 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.117731 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.117737 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.117939 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.118723 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:55.118728 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:55.118942 (+ 214us) client_negotiation.cc:769] Sending connection context
0929 15:51:55.119121 (+ 179us) client_negotiation.cc:240] Negotiation successful
0929 15:51:55.119490 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1992,"thread_start_us":1818,"threads_started":1}
I20260929 15:51:55.120064 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.098566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51194 (local address 127.15.196.254:44195)
0929 15:51:55.100118 (+ 1552us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:55.100125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:55.100883 (+ 758us) server_negotiation.cc:407] Connection header received
0929 15:51:55.101222 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:55.101229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:55.101320 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:55.101438 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:55.103182 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.104310 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.106513 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.107207 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.107331 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.112912 (+ 5581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:55.112918 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:55.112923 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:55.112963 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:55.116399 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.117393 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.117399 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.117402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.117585 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.118089 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.118094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.118097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.118511 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:55.118644 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:55.119274 (+ 630us) server_negotiation.cc:299] Negotiation successful
0929 15:51:55.119546 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1406}
I20260929 15:51:55.121372 23023 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:51:55 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697115, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.252@KRBTEST.COM for oryx/127.15.196.253@KRBTEST.COM
I20260929 15:51:55.133832 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.122057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:45983 (local address 127.0.0.1:48534)
0929 15:51:55.122245 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:55.122263 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:55.122352 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:55.122612 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:55.122615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:55.122898 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:55.123040 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.123047 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.124094 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.124097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.124982 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.124992 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.125661 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.125666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.125796 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.126401 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:55.126420 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:55.128629 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:55.131864 (+ 3235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.131868 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.131871 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.132236 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.132588 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.132591 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.132593 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.132714 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.133174 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:55.133177 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:55.133289 (+ 112us) client_negotiation.cc:769] Sending connection context
0929 15:51:55.133391 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 15:51:55.133609 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 15:51:55.133910 22979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.122170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48534 (local address 127.15.196.253:45983)
0929 15:51:55.122332 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:55.122336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:55.122353 (+ 17us) server_negotiation.cc:407] Connection header received
0929 15:51:55.122430 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:55.122434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:55.122501 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:55.122581 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:55.123176 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.123978 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.125108 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.125502 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.125595 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.128815 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:55.128821 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:55.128825 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:55.128858 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:55.131745 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.132365 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.132369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.132372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.132492 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.132803 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.132806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.132808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.133082 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:55.133190 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:55.133474 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 15:51:55.133646 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 15:51:55.134865 23023 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:51:55 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697115, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.252@KRBTEST.COM for oryx/127.15.196.252@KRBTEST.COM
I20260929 15:51:55.146322 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.135559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:34407 (local address 127.0.0.1:41772)
0929 15:51:55.135755 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:55.135770 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:55.135844 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:55.136096 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:55.136099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:55.136391 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:55.136535 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.136542 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.137495 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.137498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.138356 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.138366 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.138985 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.138989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.139131 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.139734 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:55.139750 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:55.141566 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:55.144416 (+ 2850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.144420 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.144422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.144758 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.145054 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.145057 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.145059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.145194 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.145627 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:55.145630 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:55.145744 (+ 114us) client_negotiation.cc:769] Sending connection context
0929 15:51:55.145846 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 15:51:55.146038 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 15:51:55.146394 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.135589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41772 (local address 127.15.196.252:34407)
0929 15:51:55.135769 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:55.135774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:55.135800 (+ 26us) server_negotiation.cc:407] Connection header received
0929 15:51:55.135934 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:55.135937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:55.135988 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:55.136063 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:55.136642 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.137309 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.138468 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.138845 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.138938 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.141735 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:55.141740 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:55.141743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:55.141791 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:55.144303 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.144852 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.144855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.144857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.144956 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.145277 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.145280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.145282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.145534 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:55.145636 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:55.145928 (+ 292us) server_negotiation.cc:299] Negotiation successful
0929 15:51:55.146163 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":55}
I20260929 15:51:55.150959 23023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: Bootstrap starting.
I20260929 15:51:55.154088 23023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:55.155009 23023 log.cc:824] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:55.157541 23023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: No bootstrap required, opened a new log
I20260929 15:51:55.164440 23023 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } }
I20260929 15:51:55.164746 23023 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:55.164829 23023 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb3483982a74bf5b724b88fee7f15e3, State: Initialized, Role: FOLLOWER
I20260929 15:51:55.165285 23023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } }
I20260929 15:51:55.166059 23029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } } }
I20260929 15:51:55.166349 23029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:51:55.167145 23023 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [sys.catalog]: configured and running, proceeding with master startup.
W20260929 15:51:55.176061 23040 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 15:51:55.176218 23040 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 15:51:55.180671 22867 master_runner.cc:428] Detected that this master 1cb3483982a74bf5b724b88fee7f15e3 is joining an existing cluster
I20260929 15:51:55.180806 22867 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.196.252:34407
I20260929 15:51:55.204384 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.185242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.15.196.254:44195)
0929 15:51:55.185469 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:55.185476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:55.185989 (+ 513us) server_negotiation.cc:407] Connection header received
0929 15:51:55.186318 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:55.186326 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:55.186428 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:55.186511 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:55.188354 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.189859 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.192260 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.192916 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.193031 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.196024 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:55.196031 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:55.196036 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:55.196092 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:55.200084 (+ 3992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.200982 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.201000 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.201005 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.201160 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.202152 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.202157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.202160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.202529 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:55.202960 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:55.203727 (+ 767us) server_negotiation.cc:299] Negotiation successful
0929 15:51:55.203970 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 15:51:55.207228 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.185067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.0.0.1:51204)
0929 15:51:55.185861 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:55.185885 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:55.186141 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:55.186765 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:55.186773 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:55.187676 (+ 903us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:55.188144 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.188160 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.190097 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.190105 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.192120 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.192130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.193292 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.193297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.193832 (+ 535us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.194571 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:55.194593 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:55.195808 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:55.200261 (+ 4453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.200268 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.200276 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.200842 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.201311 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.201316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.201320 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.202015 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.203059 (+ 1044us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:55.203069 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:55.203371 (+ 302us) client_negotiation.cc:769] Sending connection context
0929 15:51:55.205670 (+ 2299us) client_negotiation.cc:240] Negotiation successful
0929 15:51:55.206007 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":168,"thread_start_us":374,"threads_started":1}
I20260929 15:51:55.210990 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.186140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41784 (local address 127.15.196.252:34407)
0929 15:51:55.186841 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:55.186850 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:55.194597 (+ 7747us) server_negotiation.cc:407] Connection header received
0929 15:51:55.194779 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:55.194784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:55.194839 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:55.194894 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:55.196514 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.197674 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.199828 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.200574 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.200672 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.203092 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:55.203098 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:55.203103 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:55.203158 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:55.206824 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.208774 (+ 1950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.208779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.208783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.208955 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.209355 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.209360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.209363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.209786 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:55.209933 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:55.210417 (+ 484us) server_negotiation.cc:299] Negotiation successful
0929 15:51:55.210643 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":536}
I20260929 15:51:55.210994 22979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.185048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48542 (local address 127.15.196.253:45983)
0929 15:51:55.185253 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:55.185262 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:55.191859 (+ 6597us) server_negotiation.cc:407] Connection header received
0929 15:51:55.192272 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:55.192277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:55.192355 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:55.192418 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:55.194207 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.195313 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.197433 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.198251 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.198351 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.201092 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:55.201100 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:55.201104 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:55.201139 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:55.206728 (+ 5589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.208531 (+ 1803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.208536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.208540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.208676 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.209219 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.209224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.209227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.209609 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:55.209735 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:55.210328 (+ 593us) server_negotiation.cc:299] Negotiation successful
0929 15:51:55.210580 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 15:51:55.211983 23048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.185902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:34407 (local address 127.0.0.1:41784)
0929 15:51:55.194509 (+ 8607us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:55.194527 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:55.194674 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:55.195088 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:55.195093 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:55.195976 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:55.196341 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.196355 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.197915 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.197920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.199670 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.199684 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.201017 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.201024 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.201201 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.201778 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:55.201818 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:55.202862 (+ 1044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:55.206982 (+ 4120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.206990 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.206994 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.207527 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.209093 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.209100 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.209108 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.209250 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.210004 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:55.210008 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:55.210152 (+ 144us) client_negotiation.cc:769] Sending connection context
0929 15:51:55.211234 (+ 1082us) client_negotiation.cc:240] Negotiation successful
0929 15:51:55.211543 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8103,"thread_start_us":7740,"threads_started":1}
I20260929 15:51:55.212682 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.185070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:45983 (local address 127.0.0.1:48542)
0929 15:51:55.191762 (+ 6692us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:55.191780 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:55.192124 (+ 344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:55.192624 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:55.192629 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:55.193287 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:55.194077 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.194094 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.195530 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.195539 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.197252 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.197272 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.198676 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.198684 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.198862 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.199629 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:55.199660 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:55.200881 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:55.207892 (+ 7011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.207897 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.207900 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.208395 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.208845 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.208852 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.208858 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.209053 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.209831 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:55.209835 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:55.210022 (+ 187us) client_negotiation.cc:769] Sending connection context
0929 15:51:55.211630 (+ 1608us) client_negotiation.cc:240] Negotiation successful
0929 15:51:55.211962 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6567,"mutex_wait_us":62,"thread_start_us":6744,"threads_started":1}
Sep 29 15:51:55 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.254@KRBTEST.COM for oryx/127.15.196.252@KRBTEST.COM
I20260929 15:51:55.221769 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 22367
I20260929 15:51:55.228773 22977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.213363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:34407 (local address 127.0.0.1:41792)
0929 15:51:55.213608 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:55.213630 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:55.213733 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:55.214242 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:55.214247 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:55.214720 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:55.214908 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.214919 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.216325 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.216329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.217229 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.217242 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.218305 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.218314 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.218455 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.219042 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:55.219059 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:55.221617 (+ 2558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:55.225594 (+ 3977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.225600 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.225604 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.226168 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.226823 (+ 655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.226830 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.226837 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.227144 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.227861 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:55.227866 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:55.228032 (+ 166us) client_negotiation.cc:769] Sending connection context
0929 15:51:55.228156 (+ 124us) client_negotiation.cc:240] Negotiation successful
0929 15:51:55.228422 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260929 15:51:55.229709 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.213733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41792 (local address 127.15.196.252:34407)
0929 15:51:55.213884 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:55.213890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:55.213906 (+ 16us) server_negotiation.cc:407] Connection header received
0929 15:51:55.213987 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:55.213992 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:55.214090 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:55.214187 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:55.215048 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.216203 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.217383 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.218127 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.218256 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.221802 (+ 3546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:55.221809 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:55.221814 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:55.221855 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:55.225435 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.226388 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.226394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.226397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.226635 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.227343 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.227348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.227351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.227716 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:55.229044 (+ 1328us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:55.229171 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 15:51:55.229377 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 15:51:55.231559 22306 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.196.252:34407: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3"
member_type: NON_VOTER
last_known_addr {
host: "127.15.196.252"
port: 34407
}
attrs {
promote: true
}
}
dest_uuid: "4a54661c127145c58a343e416a1c63da"
cas_config_opid_index: 5
I20260929 15:51:55.232882 22306 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: NON_VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: true } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } }
Sep 29 15:51:55 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.196.193@KRBTEST.COM
I20260929 15:51:55.235029 22975 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1cb3483982a74bf5b724b88fee7f15e3 (127.15.196.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: NON_VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: true } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } } }
I20260929 15:51:55.236042 22998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:55.238039 22914 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 LEARNER]: Refusing update from remote peer 4a54661c127145c58a343e416a1c63da: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 15:51:55.241617 22998 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 1 FOLLOWER]: Refusing update from remote peer 4a54661c127145c58a343e416a1c63da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 15:51:55.245649 23049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.222780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43738 (local address 127.15.196.193:37331)
0929 15:51:55.223434 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:55.223441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:55.223464 (+ 23us) server_negotiation.cc:407] Connection header received
0929 15:51:55.223523 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:55.223529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:55.223837 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:55.224006 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:55.225222 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.226783 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.228691 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.229606 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.229741 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.236604 (+ 6863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:55.236621 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:55.236642 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:55.236700 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:55.240888 (+ 4188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.241974 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.241983 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.241988 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.242144 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.242822 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.242827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.242830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.243308 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:55.244774 (+ 1466us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:55.244939 (+ 165us) server_negotiation.cc:299] Negotiation successful
0929 15:51:55.245219 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":305,"threads_started":1}
I20260929 15:51:55.247139 22490 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:55.250752 23052 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: NON_VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260929 15:51:55.251575 22975 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.196.252:34407
I20260929 15:51:55.252120 23050 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:55.252318 22867 master.cc:537] Master@127.15.196.252:34407 shutting down...
I20260929 15:51:55.274737 22914 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1cb3483982a74bf5b724b88fee7f15e3 (127.15.196.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: NON_VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: true } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } } }
I20260929 15:51:55.282248 22942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: NON_VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: true } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } } }
I20260929 15:51:55.282563 22942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: This master's current role is: LEARNER
I20260929 15:51:55.287942 23051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: NON_VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: true } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } } }
I20260929 15:51:55.288355 23051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:55.289096 23051 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da: attempting to promote NON_VOTER 5f752fb87cd64afc99afd30cd9da9cee to VOTER
I20260929 15:51:55.289911 23051 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } }
I20260929 15:51:55.291671 22975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: NON_VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: true } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } } } pending_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } } }
I20260929 15:51:55.292164 22975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:55.292897 22914 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 LEARNER]: Refusing update from remote peer 4a54661c127145c58a343e416a1c63da: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260929 15:51:55.293499 23054 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260929 15:51:55.294370 22867 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:55.294783 22867 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:55.294992 22867 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: stopping tablet replica
I20260929 15:51:55.296056 23051 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5f752fb87cd64afc99afd30cd9da9cee (127.15.196.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } } }
I20260929 15:51:55.296732 22914 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5f752fb87cd64afc99afd30cd9da9cee (127.15.196.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } } }
I20260929 15:51:55.300709 23054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } } }
I20260929 15:51:55.301081 23054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:55.301353 22942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } } }
I20260929 15:51:55.301608 22942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:51:55.301921 23051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } } }
I20260929 15:51:55.302148 23051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:55.303325 22977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.302754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:34407 (local address 127.0.0.1:41806)
0929 15:51:55.302980 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:55.303127 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:34407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260929 15:51:55.303565 22292 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da -> Peer 1cb3483982a74bf5b724b88fee7f15e3 (127.15.196.252:34407): Couldn't send request to peer 1cb3483982a74bf5b724b88fee7f15e3. Status: Network error: Client connection negotiation failed: client connection to 127.15.196.252:34407: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 15:51:55.304993 22867 master.cc:552] Master@127.15.196.252:34407 shutdown complete.
I20260929 15:51:55.305100 22867 master_runner.cc:305] Clearing existing system tablet
I20260929 15:51:55.305514 22867 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:55.305655 22867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:55.308573 22867 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260929 15:51:55.315687 23059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:55.316100 22867 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:55.316534 22867 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.007s
I20260929 15:51:55.316684 22867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/wal
uuid: "1cb3483982a74bf5b724b88fee7f15e3"
format_stamp: "Formatted at 2026-09-29 15:51:55 on dist-test-slave-c51v"
server_key: "c1eee3d85e582cf35c86edde80b4804e"
server_key_iv: "429b04da8f9dd289d592ebf04f693ebf"
server_key_version: "encryptionkey@0"
I20260929 15:51:55.322156 22867 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:55.329452 22867 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 15:51:55.329783 22867 log.cc:1196] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260929 15:51:55.330341 22867 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: Deleting consensus metadata
I20260929 15:51:55.330641 22867 master_runner.cc:315] Copying system tablet from 127.15.196.254:44195
I20260929 15:51:55.333297 22867 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: tablet copy: Beginning tablet copy session from remote peer at address 127.15.196.254:44195
I20260929 15:51:55.349365 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.334494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51206 (local address 127.15.196.254:44195)
0929 15:51:55.334754 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:55.334760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:55.334900 (+ 140us) server_negotiation.cc:407] Connection header received
0929 15:51:55.335170 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:55.335176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:55.335260 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:55.335402 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:55.336374 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.337488 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.339530 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.340176 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.340272 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.342797 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:55.342804 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:55.342809 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:55.342848 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:55.346422 (+ 3574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.347197 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.347203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.347207 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.347397 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.347810 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.347815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.347819 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.348159 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:55.348312 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:55.348770 (+ 458us) server_negotiation.cc:299] Negotiation successful
0929 15:51:55.349006 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260929 15:51:55.349713 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.334163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.0.0.1:51206)
0929 15:51:55.334785 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:55.334805 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:55.335014 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:55.335435 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:55.335440 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:55.335963 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:55.336210 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.336222 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.337663 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.337669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.339215 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.339230 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.340530 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.340536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.340732 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.341464 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:55.341499 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:55.342580 (+ 1081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:55.346573 (+ 3993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.346579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.346583 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.347069 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.347521 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.347527 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.347531 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.347699 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.348301 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:55.348306 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:55.348512 (+ 206us) client_negotiation.cc:769] Sending connection context
0929 15:51:55.348656 (+ 144us) client_negotiation.cc:240] Negotiation successful
0929 15:51:55.348961 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":595,"threads_started":1}
I20260929 15:51:55.350426 22326 tablet_copy_service.cc:140] P 4a54661c127145c58a343e416a1c63da: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1cb3483982a74bf5b724b88fee7f15e3 ({username='oryx', principal='oryx/127.15.196.252@KRBTEST.COM'} at 127.0.0.1:51206)
I20260929 15:51:55.350606 22326 tablet_copy_service.cc:161] P 4a54661c127145c58a343e416a1c63da: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1cb3483982a74bf5b724b88fee7f15e3 at {username='oryx', principal='oryx/127.15.196.252@KRBTEST.COM'} at 127.0.0.1:51206: session id = 1cb3483982a74bf5b724b88fee7f15e3-00000000000000000000000000000000
I20260929 15:51:55.354295 22326 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 15:51:55.356267 22867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 15:51:55.363327 22867 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: tablet copy: Starting download of 0 data blocks...
I20260929 15:51:55.363643 22867 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: tablet copy: Starting download of 1 WAL segments...
I20260929 15:51:55.366269 22867 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 15:51:55.370882 22326 tablet_copy_service.cc:342] P 4a54661c127145c58a343e416a1c63da: Request end of tablet copy session 1cb3483982a74bf5b724b88fee7f15e3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.196.252@KRBTEST.COM'} at 127.0.0.1:51206
I20260929 15:51:55.371034 22326 tablet_copy_service.cc:433] P 4a54661c127145c58a343e416a1c63da: ending tablet copy session 1cb3483982a74bf5b724b88fee7f15e3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1cb3483982a74bf5b724b88fee7f15e3
I20260929 15:51:55.384980 22867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:55.387766 23066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:55.392759 23067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:55.397245 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 22367
W20260929 15:51:55.400494 23069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:55.403666 22867 server_base.cc:1034] running on GCE node
I20260929 15:51:55.404430 22867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:55.405210 22867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:55.406409 22867 hybrid_clock.cc:648] HybridClock initialized: now 1790697115406376 us; error 62 us; skew 500 ppm
I20260929 15:51:55.406656 22867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:55 dist-test-slave-c51v krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697115, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:55.409065 22867 init.cc:377] Logged in from keytab as oryx/127.15.196.252@KRBTEST.COM (short username oryx)
I20260929 15:51:55.416326 22867 webserver.cc:460] Webserver started at http://127.15.196.252:34627/ using document root <none> and password file <none>
I20260929 15:51:55.416684 22867 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:55.416778 22867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:55.421542 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:37331
--local_ip_for_outbound_sockets=127.15.196.193
--tserver_master_addrs=127.15.196.254:44195
--webserver_port=46575
--webserver_interface=127.15.196.193
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.193.keytab
--principal=oryx/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.196.254:44195,127.15.196.253:45983
--tserver_master_addrs=127.15.196.254:44195,127.15.196.253:45983,127.15.196.252:34407 with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:51:55.426436 22867 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.002s sys 0.005s
I20260929 15:51:55.428557 23076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:55.429054 22867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 15:51:55.429179 22867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/wal
uuid: "1cb3483982a74bf5b724b88fee7f15e3"
format_stamp: "Formatted at 2026-09-29 15:51:55 on dist-test-slave-c51v"
server_key: "c1eee3d85e582cf35c86edde80b4804e"
server_key_iv: "429b04da8f9dd289d592ebf04f693ebf"
server_key_version: "encryptionkey@0"
I20260929 15:51:55.429363 22867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:55.447207 22867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:55.451438 22867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:55.479499 22867 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.252:34407
I20260929 15:51:55.479569 23128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.252:34407 every 8 connection(s)
I20260929 15:51:55.480544 22867 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/master-2/data/info.pb
I20260929 15:51:55.486176 23129 sys_catalog.cc:263] Verifying existing consensus state
I20260929 15:51:55.490875 23129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: Bootstrap starting.
I20260929 15:51:55.509065 23129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 15:51:55.509584 23129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: Bootstrap complete.
I20260929 15:51:55.510731 23129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } }
I20260929 15:51:55.511072 23129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1cb3483982a74bf5b724b88fee7f15e3, State: Initialized, Role: LEARNER
I20260929 15:51:55.511368 23129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } }
I20260929 15:51:55.512198 23132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } } }
I20260929 15:51:55.512511 23129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:51:55.512650 23132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [sys.catalog]: This master's current role is: LEARNER
I20260929 15:51:55.521358 23143 catalog_manager.cc:1259] Loaded cluster ID: 894250600ca246a284b71fbfe3c54beb
I20260929 15:51:55.521449 23143 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: loading cluster ID for follower catalog manager: success
I20260929 15:51:55.524776 23143 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: acquiring CA information for follower catalog manager: success
I20260929 15:51:55.527227 23143 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 15:51:55.789009 23080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:55.789397 23080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:55.789556 23080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:55.796855 23080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:55.797013 23080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:55.797180 23080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:51:55.807257 23080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.193.keytab
--principal=oryx/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:37331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=46575
--tserver_master_addrs=127.15.196.254:44195,127.15.196.253:45983,127.15.196.252:34407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:55.808686 23080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:55.810004 23080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:55.823841 23151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:55.823843 23149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:55.823910 23148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:55.824556 23080 server_base.cc:1034] running on GCE node
I20260929 15:51:55.825214 23080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:55.826181 23080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:55.827389 23080 hybrid_clock.cc:648] HybridClock initialized: now 1790697115827328 us; error 42 us; skew 500 ppm
I20260929 15:51:55.827787 23080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:55 dist-test-slave-c51v krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697115, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:55.831158 23080 init.cc:377] Logged in from keytab as oryx/127.15.196.193@KRBTEST.COM (short username oryx)
I20260929 15:51:55.832656 23080 webserver.cc:460] Webserver started at http://127.15.196.193:46575/ using document root <none> and password file <none>
I20260929 15:51:55.833361 23080 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:55.833524 23080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:55.838936 23080 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:51:55.841761 23158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:55.843156 23080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:51:55.843385 23080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "94640710d80f4186bd98dcc97d749008"
format_stamp: "Formatted at 2026-09-29 15:51:52 on dist-test-slave-c51v"
server_key: "ac11dcb2bfef1beb0c5da8ee3de069c4"
server_key_iv: "e70efaacb0363e09a5f65c0b706c56cc"
server_key_version: "encryptionkey@0"
I20260929 15:51:55.844400 23080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:55.861042 23080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:55.865051 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.859270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41822 (local address 127.15.196.252:34407)
0929 15:51:55.859936 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:55.859941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:55.859967 (+ 26us) server_negotiation.cc:407] Connection header received
0929 15:51:55.860051 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:55.860055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:55.860124 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:55.860251 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:55.860661 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.861658 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.863130 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.864032 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.864230 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.864419 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:55.864676 (+ 257us) server_negotiation.cc:299] Negotiation successful
0929 15:51:55.864775 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":350,"threads_started":1}
I20260929 15:51:55.865128 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.859123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:34407 (local address 127.0.0.1:41822)
0929 15:51:55.859633 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:55.859656 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:55.859802 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:55.860275 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:55.860280 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:55.860310 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:55.860510 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.860520 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.861809 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.861814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.862997 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.863013 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.864249 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.864257 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.864494 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.864546 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 15:51:55.864660 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 15:51:55.864810 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":273,"threads_started":1}
I20260929 15:51:55.866045 23104 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260929 15:51:55.866221 23080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:55.866520 23080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:55.866734 23132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a54661c127145c58a343e416a1c63da. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } } }
I20260929 15:51:55.867069 23132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [sys.catalog]: This master's current role is: LEARNER
I20260929 15:51:55.867725 23080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:55.869047 23080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:55.869132 23080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:55.869211 23080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:55.869269 23080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:55.902906 23080 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:37331
I20260929 15:51:55.902961 23272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:37331 every 8 connection(s)
I20260929 15:51:55.904491 23080 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:51:55.913338 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 23080
I20260929 15:51:55.913753 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 22501
Sep 29 15:51:55 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697115, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.252@KRBTEST.COM
Sep 29 15:51:55 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697115, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.253@KRBTEST.COM
Sep 29 15:51:55 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697115, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM
I20260929 15:51:55.934055 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.907561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:54775 (local address 127.15.196.253:45983)
0929 15:51:55.907976 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:55.907981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:55.909284 (+ 1303us) server_negotiation.cc:407] Connection header received
0929 15:51:55.911816 (+ 2532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:55.911821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:55.911920 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:55.912050 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:55.914681 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.916114 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.919671 (+ 3557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.920603 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.920732 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.926724 (+ 5992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:55.926729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:55.926732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:55.926765 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:55.930258 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.931120 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.931125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.931129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.931256 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.931850 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.931858 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.931863 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.932211 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:55.932300 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:55.933390 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0929 15:51:55.933643 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":876,"threads_started":1}
I20260929 15:51:55.935438 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.907537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:33979 (local address 127.15.196.252:34407)
0929 15:51:55.907742 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:55.907750 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:55.909814 (+ 2064us) server_negotiation.cc:407] Connection header received
0929 15:51:55.911942 (+ 2128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:55.911948 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:55.912061 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:55.912149 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:55.914651 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.916341 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.919674 (+ 3333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.920538 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.920669 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.926726 (+ 6057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:55.926733 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:55.926738 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:55.927176 (+ 438us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:55.932297 (+ 5121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.933077 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.933084 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.933087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.933230 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.933663 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.933668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.933671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.934009 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:55.934158 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:55.934884 (+ 726us) server_negotiation.cc:299] Negotiation successful
0929 15:51:55.935102 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 15:51:55.936908 23278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.908122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:45983 (local address 127.15.196.193:54775)
0929 15:51:55.909123 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:55.909172 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:55.911550 (+ 2378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:55.912291 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:55.912299 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:55.913278 (+ 979us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:55.914446 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.914474 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.916398 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.916406 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.919454 (+ 3048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.919475 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.921123 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.921132 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.921419 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.922486 (+ 1067us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:55.922532 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:55.926433 (+ 3901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:55.930428 (+ 3995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.930440 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.930458 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.930995 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.931436 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.931441 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.931444 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.931704 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.932412 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:55.932421 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:55.932912 (+ 491us) client_negotiation.cc:769] Sending connection context
0929 15:51:55.934534 (+ 1622us) client_negotiation.cc:240] Negotiation successful
0929 15:51:55.935118 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"mutex_wait_us":290,"spinlock_wait_cycles":896,"thread_start_us":915,"threads_started":1}
I20260929 15:51:55.936885 23279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.908122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:34407 (local address 127.15.196.193:33979)
0929 15:51:55.909123 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:55.909723 (+ 600us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:55.911568 (+ 1845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:55.912335 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:55.912340 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:55.913291 (+ 951us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:55.914447 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.914472 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.916559 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.916568 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.919460 (+ 2892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.919479 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.921036 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.921043 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.921348 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.922197 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:55.922232 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:55.926441 (+ 4209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:55.932420 (+ 5979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.932424 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.932427 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.932917 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.933358 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.933363 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.933367 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.933532 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.934228 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:55.934232 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:55.934364 (+ 132us) client_negotiation.cc:769] Sending connection context
0929 15:51:55.934534 (+ 170us) client_negotiation.cc:240] Negotiation successful
0929 15:51:55.935919 (+ 1385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":1502,"spinlock_wait_cycles":47616,"thread_start_us":412,"threads_started":1}
I20260929 15:51:55.938638 23275 heartbeater.cc:347] Connected to a master server at 127.15.196.253:45983
I20260929 15:51:55.938669 23273 heartbeater.cc:347] Connected to a master server at 127.15.196.252:34407
I20260929 15:51:55.938927 23275 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:55.938933 23273 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:55.939828 23275 heartbeater.cc:511] Master 127.15.196.253:45983 requested a full tablet report, sending...
I20260929 15:51:55.939851 23273 heartbeater.cc:511] Master 127.15.196.252:34407 requested a full tablet report, sending...
I20260929 15:51:55.942026 22904 ts_manager.cc:194] Registered new tserver with Master: 94640710d80f4186bd98dcc97d749008 (127.15.196.193:37331)
I20260929 15:51:55.942080 23094 ts_manager.cc:194] Registered new tserver with Master: 94640710d80f4186bd98dcc97d749008 (127.15.196.193:37331)
I20260929 15:51:55.944664 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.909720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.193:57563)
0929 15:51:55.910466 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:55.910486 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:55.911568 (+ 1082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:55.912345 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:55.912349 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:55.913291 (+ 942us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:55.920569 (+ 7278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.920584 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.922922 (+ 2338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.922927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.925146 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:55.925161 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.926215 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.926224 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:55.926411 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.926997 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:55.927119 (+ 122us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:55.931944 (+ 4825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:55.937003 (+ 5059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.937008 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.937011 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.937452 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.942879 (+ 5427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:55.942884 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:55.942889 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:55.943015 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:55.943850 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:55.943856 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:55.944052 (+ 196us) client_negotiation.cc:769] Sending connection context
0929 15:51:55.944158 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 15:51:55.944369 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":108,"spinlock_wait_cycles":37760,"thread_start_us":421,"threads_started":1}
I20260929 15:51:55.945324 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.910200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:57563 (local address 127.15.196.254:44195)
0929 15:51:55.910700 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:55.910706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:55.910722 (+ 16us) server_negotiation.cc:407] Connection header received
0929 15:51:55.911923 (+ 1201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:55.911928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:55.912003 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:55.912109 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:55.920752 (+ 8643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.922466 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.925321 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.926001 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.926147 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.932150 (+ 6003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:55.932158 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:55.932163 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:55.932209 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:55.935883 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.939735 (+ 3852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.939743 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.939748 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.941742 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.943252 (+ 1510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.943259 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.943262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.943693 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:55.943898 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:55.944327 (+ 429us) server_negotiation.cc:299] Negotiation successful
0929 15:51:55.944606 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":259,"threads_started":1}
I20260929 15:51:55.946147 23274 heartbeater.cc:347] Connected to a master server at 127.15.196.254:44195
I20260929 15:51:55.946293 23274 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:55.946688 23274 heartbeater.cc:511] Master 127.15.196.254:44195 requested a full tablet report, sending...
Sep 29 15:51:55 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.196.194@KRBTEST.COM
I20260929 15:51:55.947893 22306 ts_manager.cc:194] Re-registered known tserver with Master: 94640710d80f4186bd98dcc97d749008 (127.15.196.193:37331)
I20260929 15:51:55.949030 22306 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:57563
I20260929 15:51:55.954314 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:55.914801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36286 (local address 127.15.196.194:33961)
0929 15:51:55.925758 (+ 10957us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:55.925767 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:55.933862 (+ 8095us) server_negotiation.cc:407] Connection header received
0929 15:51:55.934430 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:55.934442 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:55.934747 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:55.934906 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:55.937353 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.939532 (+ 2179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.941686 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:55.942540 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:55.942665 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:55.948631 (+ 5966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:55.948647 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:55.948665 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:55.948706 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:55.951922 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.952556 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.952562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.952565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.952692 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:55.953001 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:55.953004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:55.953006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:55.953302 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:55.953523 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:55.953730 (+ 207us) server_negotiation.cc:299] Negotiation successful
0929 15:51:55.953915 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10752,"thread_start_us":10961,"threads_started":1}
I20260929 15:51:55.955687 22624 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:56.040707 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 22501
I20260929 15:51:56.057502 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:33961
--local_ip_for_outbound_sockets=127.15.196.194
--tserver_master_addrs=127.15.196.254:44195
--webserver_port=36479
--webserver_interface=127.15.196.194
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.194.keytab
--principal=oryx/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.196.254:44195,127.15.196.253:45983
--tserver_master_addrs=127.15.196.254:44195,127.15.196.253:45983,127.15.196.252:34407 with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:51:56.296844 23051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: NON_VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: true } } }
I20260929 15:51:56.297254 23051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:56.310599 23286 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da: attempting to promote NON_VOTER 1cb3483982a74bf5b724b88fee7f15e3 to VOTER
I20260929 15:51:56.311456 23286 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } }
I20260929 15:51:56.314137 23104 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 1 LEARNER]: Refusing update from remote peer 4a54661c127145c58a343e416a1c63da: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260929 15:51:56.314245 22914 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 FOLLOWER]: Refusing update from remote peer 4a54661c127145c58a343e416a1c63da: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260929 15:51:56.315145 23287 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260929 15:51:56.315651 23286 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260929 15:51:56.320488 23286 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 1cb3483982a74bf5b724b88fee7f15e3 (127.15.196.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } } }
I20260929 15:51:56.324095 23104 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 1cb3483982a74bf5b724b88fee7f15e3 (127.15.196.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } } }
I20260929 15:51:56.330927 23286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } } }
I20260929 15:51:56.331053 23051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } } }
I20260929 15:51:56.331395 23051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:56.331589 23286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:56.332168 23132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } } }
I20260929 15:51:56.332465 23132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:51:56.333813 23287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } } }
I20260929 15:51:56.334151 23287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:56.334645 22914 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 1cb3483982a74bf5b724b88fee7f15e3 (127.15.196.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } } }
I20260929 15:51:56.339155 23293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4a54661c127145c58a343e416a1c63da" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } } }
I20260929 15:51:56.339639 23293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: This master's current role is: FOLLOWER
W20260929 15:51:56.409881 23285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:56.410193 23285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:56.410354 23285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:56.417980 23285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:56.418118 23285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:56.418360 23285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:51:56.428826 23285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.194.keytab
--principal=oryx/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:33961
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=36479
--tserver_master_addrs=127.15.196.254:44195,127.15.196.253:45983,127.15.196.252:34407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:56.430363 23285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:56.431851 23285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:56.447657 23298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:56.447635 23299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:56.448557 23301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:56.448796 23285 server_base.cc:1034] running on GCE node
I20260929 15:51:56.450140 23285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:56.451208 23285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:56.452443 23285 hybrid_clock.cc:648] HybridClock initialized: now 1790697116452375 us; error 68 us; skew 500 ppm
I20260929 15:51:56.452765 23285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:56 dist-test-slave-c51v krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697116, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:56.456300 23285 init.cc:377] Logged in from keytab as oryx/127.15.196.194@KRBTEST.COM (short username oryx)
I20260929 15:51:56.458103 23285 webserver.cc:460] Webserver started at http://127.15.196.194:36479/ using document root <none> and password file <none>
I20260929 15:51:56.458793 23285 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:56.458959 23285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:56.464454 23285 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:51:56.467252 23308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:56.468807 23285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:51:56.469255 23285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "5cf3ae47c7cb4f09a336e751fa90261f"
format_stamp: "Formatted at 2026-09-29 15:51:53 on dist-test-slave-c51v"
server_key: "02d0eb95d251e430f8135b9acaea0e69"
server_key_iv: "848c947f992e65ee85c132c6d4a25cb7"
server_key_version: "encryptionkey@0"
I20260929 15:51:56.470397 23285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:56.488189 23285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:56.492614 23285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:56.492956 23285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:56.494082 23285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:56.495637 23285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:56.495757 23285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:56.495898 23285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:56.495998 23285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:56.533679 23285 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:33961
I20260929 15:51:56.533728 23420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:33961 every 8 connection(s)
I20260929 15:51:56.535159 23285 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:51:56.538426 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 23285
I20260929 15:51:56.538770 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 22634
Sep 29 15:51:56 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697116, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.252@KRBTEST.COM
Sep 29 15:51:56 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697116, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.253@KRBTEST.COM
I20260929 15:51:56.568964 23426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:56.539362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:57803 (local address 127.15.196.252:34407)
0929 15:51:56.544266 (+ 4904us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:56.544275 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:56.544301 (+ 26us) server_negotiation.cc:407] Connection header received
0929 15:51:56.544379 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:56.544388 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:56.544495 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:56.547390 (+ 2895us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:56.547686 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:56.549234 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:56.552692 (+ 3458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:56.553626 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:56.553776 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:56.560293 (+ 6517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:56.560304 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:56.560313 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:56.560377 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:56.564343 (+ 3966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:56.565400 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:56.565405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:56.565408 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:56.565555 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:56.566095 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:56.566099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:56.566102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:56.566460 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:56.566578 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:56.568283 (+ 1705us) server_negotiation.cc:299] Negotiation successful
0929 15:51:56.568539 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4738,"thread_start_us":4596,"threads_started":1}
I20260929 15:51:56.570475 23425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:56.538705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:34407 (local address 127.15.196.194:57803)
0929 15:51:56.539628 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:56.539675 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:56.542446 (+ 2771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:56.545013 (+ 2567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:56.545030 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:56.546048 (+ 1018us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:56.547120 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:56.547150 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:56.549453 (+ 2303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:56.549462 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:56.552406 (+ 2944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:56.552434 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:56.554189 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:56.554199 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:56.554644 (+ 445us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:56.555669 (+ 1025us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:56.555711 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:56.559925 (+ 4214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:56.564541 (+ 4616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:56.564552 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:56.564571 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:56.565227 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:56.565693 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:56.565698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:56.565701 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:56.565954 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:56.566659 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:56.566668 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:56.567195 (+ 527us) client_negotiation.cc:769] Sending connection context
0929 15:51:56.567570 (+ 375us) client_negotiation.cc:240] Negotiation successful
0929 15:51:56.569033 (+ 1463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":2455,"threads_started":1}
Sep 29 15:51:56 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697116, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM
I20260929 15:51:56.572649 23421 heartbeater.cc:347] Connected to a master server at 127.15.196.252:34407
I20260929 15:51:56.572986 23421 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:56.573860 23421 heartbeater.cc:511] Master 127.15.196.252:34407 requested a full tablet report, sending...
Sep 29 15:51:56 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.196.195@KRBTEST.COM
I20260929 15:51:56.575613 23094 ts_manager.cc:194] Registered new tserver with Master: 5cf3ae47c7cb4f09a336e751fa90261f (127.15.196.194:33961)
I20260929 15:51:56.581185 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:56.540797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:56321 (local address 127.15.196.253:45983)
0929 15:51:56.560530 (+ 19733us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:56.560539 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:56.560564 (+ 25us) server_negotiation.cc:407] Connection header received
0929 15:51:56.560655 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:56.560663 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:56.560772 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:56.562610 (+ 1838us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:56.562771 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:56.563954 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:56.565889 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:56.566533 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:56.567277 (+ 744us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:56.570582 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:56.570588 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:56.570593 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:56.570638 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:56.578439 (+ 7801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:56.579082 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:56.579089 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:56.579092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:56.579239 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:56.579672 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:56.579675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:56.579677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:56.579938 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:56.580103 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:56.580575 (+ 472us) server_negotiation.cc:299] Negotiation successful
0929 15:51:56.580810 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19550,"thread_start_us":19373,"threads_started":1}
I20260929 15:51:56.581391 23427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:56.539680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:45983 (local address 127.15.196.194:56321)
0929 15:51:56.543416 (+ 3736us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:56.543441 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:56.543585 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:56.561571 (+ 17986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:56.561575 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:56.562152 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:56.562457 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:56.562468 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:56.564139 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:56.564146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:56.565695 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:56.565707 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:56.566742 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:56.566748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:56.566939 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:56.567584 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:56.567605 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:56.570366 (+ 2761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:56.578577 (+ 8211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:56.578583 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:56.578587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:56.578949 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:56.579409 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:56.579413 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:56.579417 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:56.579562 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:56.580077 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:56.580081 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:56.580262 (+ 181us) client_negotiation.cc:769] Sending connection context
0929 15:51:56.580441 (+ 179us) client_negotiation.cc:240] Negotiation successful
0929 15:51:56.580767 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3533,"spinlock_wait_cycles":896,"thread_start_us":3311,"threads_started":1}
I20260929 15:51:56.581615 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:56.542287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.194:59217)
0929 15:51:56.561222 (+ 18935us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:56.561245 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:56.561379 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:56.563402 (+ 2023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:56.563407 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:56.563908 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:56.564294 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:56.564304 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:56.565687 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:56.565692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:56.567280 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:56.567311 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:56.568474 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:56.568480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:56.568659 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:56.569246 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:56.569268 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:56.573238 (+ 3970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:56.577581 (+ 4343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:56.577586 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:56.577590 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:56.578120 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:56.579105 (+ 985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:56.579109 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:56.579112 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:56.579254 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:56.580228 (+ 974us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:56.580233 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:56.580367 (+ 134us) client_negotiation.cc:769] Sending connection context
0929 15:51:56.580484 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 15:51:56.580874 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18813,"mutex_wait_us":145,"thread_start_us":18684,"threads_started":1}
I20260929 15:51:56.582401 23423 heartbeater.cc:347] Connected to a master server at 127.15.196.253:45983
I20260929 15:51:56.582504 23432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:56.542438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:59217 (local address 127.15.196.254:44195)
0929 15:51:56.557769 (+ 15331us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:56.557777 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:56.561362 (+ 3585us) server_negotiation.cc:407] Connection header received
0929 15:51:56.561526 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:56.561534 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:56.562569 (+ 1035us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:56.563227 (+ 658us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:56.564466 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:56.565523 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:56.567462 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:56.568128 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:56.568243 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:56.573478 (+ 5235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:56.573485 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:56.573491 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:56.573537 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:56.577279 (+ 3742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:56.578295 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:56.578302 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:56.578305 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:56.578470 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:56.579719 (+ 1249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:56.579724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:56.579728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:56.580092 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:56.580264 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:56.581534 (+ 1270us) server_negotiation.cc:299] Negotiation successful
0929 15:51:56.582154 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15169,"thread_start_us":15340,"threads_started":1}
I20260929 15:51:56.582566 23423 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:56.582955 23423 heartbeater.cc:511] Master 127.15.196.253:45983 requested a full tablet report, sending...
I20260929 15:51:56.583243 23422 heartbeater.cc:347] Connected to a master server at 127.15.196.254:44195
I20260929 15:51:56.583400 23422 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:56.583811 23422 heartbeater.cc:511] Master 127.15.196.254:44195 requested a full tablet report, sending...
I20260929 15:51:56.584234 22904 ts_manager.cc:194] Registered new tserver with Master: 5cf3ae47c7cb4f09a336e751fa90261f (127.15.196.194:33961)
I20260929 15:51:56.584761 22306 ts_manager.cc:194] Re-registered known tserver with Master: 5cf3ae47c7cb4f09a336e751fa90261f (127.15.196.194:33961)
I20260929 15:51:56.585304 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:56.540455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51412 (local address 127.15.196.195:33907)
0929 15:51:56.562327 (+ 21872us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:56.562339 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:56.562373 (+ 34us) server_negotiation.cc:407] Connection header received
0929 15:51:56.562444 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:56.562453 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:56.562735 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:56.562903 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:56.565160 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:56.566691 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:56.568843 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:56.570045 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:56.570174 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:56.576697 (+ 6523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:56.576712 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:56.576735 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:56.576771 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:56.580486 (+ 3715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:56.581365 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:56.581376 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:56.581382 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:56.581543 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:56.581990 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:56.581996 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:56.582000 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:56.582421 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:56.584495 (+ 2074us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:56.584658 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 15:51:56.584930 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21694,"thread_start_us":21551,"threads_started":1}
I20260929 15:51:56.585886 22306 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:59217
I20260929 15:51:56.586678 22757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:56.671090 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 22634
I20260929 15:51:56.687729 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:33907
--local_ip_for_outbound_sockets=127.15.196.195
--tserver_master_addrs=127.15.196.254:44195
--webserver_port=42519
--webserver_interface=127.15.196.195
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.195.keytab
--principal=oryx/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.196.254:44195,127.15.196.253:45983
--tserver_master_addrs=127.15.196.254:44195,127.15.196.253:45983,127.15.196.252:34407 with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:51:56.951244 23274 heartbeater.cc:503] Master 127.15.196.254:44195 was elected leader, sending a full tablet report...
W20260929 15:51:57.020007 23434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:51:57.020452 23434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:51:57.020645 23434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:51:57.028180 23434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:51:57.028293 23434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:51:57.028411 23434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:51:57.038506 23434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.195.keytab
--principal=oryx/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:33907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=42519
--tserver_master_addrs=127.15.196.254:44195,127.15.196.253:45983,127.15.196.252:34407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:51:57.039894 23434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:51:57.041167 23434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:51:57.054499 23439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:57.054517 23440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:51:57.054543 23442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:51:57.055899 23434 server_base.cc:1034] running on GCE node
I20260929 15:51:57.056643 23434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:51:57.057658 23434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:51:57.058884 23434 hybrid_clock.cc:648] HybridClock initialized: now 1790697117058818 us; error 52 us; skew 500 ppm
I20260929 15:51:57.059235 23434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:51:57 dist-test-slave-c51v krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697117, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:51:57.062932 23434 init.cc:377] Logged in from keytab as oryx/127.15.196.195@KRBTEST.COM (short username oryx)
I20260929 15:51:57.064327 23434 webserver.cc:460] Webserver started at http://127.15.196.195:42519/ using document root <none> and password file <none>
I20260929 15:51:57.064986 23434 fs_manager.cc:359] Metadata directory not provided
I20260929 15:51:57.065099 23434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:51:57.070263 23434 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260929 15:51:57.072908 23449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:57.074368 23434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 15:51:57.074535 23434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "d2e4767290964f7fa24e766c3c1248b7"
format_stamp: "Formatted at 2026-09-29 15:51:53 on dist-test-slave-c51v"
server_key: "e087f5a0d4d59e4e96ddba31aa3b2afe"
server_key_iv: "55eba561429b2fe3b7d48e03770bad01"
server_key_version: "encryptionkey@0"
I20260929 15:51:57.075619 23434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:51:57.098112 23434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:51:57.102717 23434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:51:57.103010 23434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:51:57.104130 23434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:51:57.105572 23434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:51:57.105695 23434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:57.105829 23434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:51:57.105918 23434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:51:57.137795 23434 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:33907
I20260929 15:51:57.137849 23561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:33907 every 8 connection(s)
I20260929 15:51:57.139362 23434 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:51:57.149516 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 23434
Sep 29 15:51:57 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697117, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.253@KRBTEST.COM
Sep 29 15:51:57 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697117, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.252@KRBTEST.COM
Sep 29 15:51:57 dist-test-slave-c51v krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697117, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM
I20260929 15:51:57.167737 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.142876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:60351 (local address 127.15.196.253:45983)
0929 15:51:57.143389 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:57.143397 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:57.143972 (+ 575us) server_negotiation.cc:407] Connection header received
0929 15:51:57.146813 (+ 2841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:57.146819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:57.146899 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:57.147000 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:57.149440 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.150582 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.153013 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.153669 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.153798 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.159978 (+ 6180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:57.159988 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:57.159996 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:57.160061 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:57.163697 (+ 3636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.164739 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.164747 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.164752 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.164920 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.165511 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.165516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.165519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.165838 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:57.165978 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:57.167105 (+ 1127us) server_negotiation.cc:299] Negotiation successful
0929 15:51:57.167352 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":228,"threads_started":1}
I20260929 15:51:57.170208 23570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.144771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:34407 (local address 127.15.196.195:33345)
0929 15:51:57.145316 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:57.145337 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:57.146432 (+ 1095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:57.147172 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:57.147181 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:57.148210 (+ 1029us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:57.149206 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:57.149234 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.150825 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.150830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:57.155722 (+ 4892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:57.155735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.156776 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.156781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:57.156940 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.157498 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:57.157516 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:57.159616 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:57.164108 (+ 4492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:57.164113 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:57.164116 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:57.164701 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:57.165086 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:57.165090 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:57.165094 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:57.165352 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:57.166054 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:57.166062 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:57.166656 (+ 594us) client_negotiation.cc:769] Sending connection context
0929 15:51:57.166979 (+ 323us) client_negotiation.cc:240] Negotiation successful
0929 15:51:57.168611 (+ 1632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":3061,"spinlock_wait_cycles":27776,"thread_start_us":280,"threads_started":1}
I20260929 15:51:57.170859 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.144854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:33345 (local address 127.15.196.252:34407)
0929 15:51:57.145768 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:57.145776 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:57.145795 (+ 19us) server_negotiation.cc:407] Connection header received
0929 15:51:57.146704 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:57.146710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:57.146792 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:57.146921 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:57.149439 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.150632 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.155908 (+ 5276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.156596 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.156747 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.159819 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:57.159829 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:57.159837 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:57.159882 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:57.163956 (+ 4074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.164829 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.164836 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.164842 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.164973 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.165498 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.165503 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.165506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.165838 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:57.165979 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:57.167460 (+ 1481us) server_negotiation.cc:299] Negotiation successful
0929 15:51:57.167684 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":799,"thread_start_us":695,"threads_started":1}
I20260929 15:51:57.172963 23562 heartbeater.cc:347] Connected to a master server at 127.15.196.252:34407
I20260929 15:51:57.173295 23562 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:57.174857 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.142173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:59745 (local address 127.15.196.254:44195)
0929 15:51:57.142651 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:57.142658 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:57.144779 (+ 2121us) server_negotiation.cc:407] Connection header received
0929 15:51:57.146813 (+ 2034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:57.146819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:57.146899 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:57.147010 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:57.149448 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.150569 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.156121 (+ 5552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.157030 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.157181 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.167249 (+ 10068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:57.167259 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:57.167267 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:57.167356 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:57.171051 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.171874 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.171880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.171883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.172047 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.172484 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.172489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.172492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.172803 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:57.172940 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:57.174274 (+ 1334us) server_negotiation.cc:299] Negotiation successful
0929 15:51:57.174544 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":225,"threads_started":1}
I20260929 15:51:57.174031 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.142786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.195:59745)
0929 15:51:57.144667 (+ 1881us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:57.144689 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:57.146414 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:57.147179 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:57.147184 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:57.148293 (+ 1109us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:57.149217 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:57.149232 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.150760 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.150765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:57.155881 (+ 5116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:57.155901 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.159585 (+ 3684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.159595 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:57.159866 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.160942 (+ 1076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:57.160972 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:57.166086 (+ 5114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:57.171214 (+ 5128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:57.171221 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:57.171225 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:57.171730 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:57.172181 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:57.172186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:57.172189 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:57.172359 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:57.173003 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:57.173007 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:57.173181 (+ 174us) client_negotiation.cc:769] Sending connection context
0929 15:51:57.173330 (+ 149us) client_negotiation.cc:240] Negotiation successful
0929 15:51:57.173608 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1704,"mutex_wait_us":3057,"spinlock_wait_cycles":90880,"thread_start_us":1389,"threads_started":1}
I20260929 15:51:57.174237 23562 heartbeater.cc:511] Master 127.15.196.252:34407 requested a full tablet report, sending...
I20260929 15:51:57.176990 23094 ts_manager.cc:194] Registered new tserver with Master: d2e4767290964f7fa24e766c3c1248b7 (127.15.196.195:33907)
I20260929 15:51:57.177167 23563 heartbeater.cc:347] Connected to a master server at 127.15.196.254:44195
I20260929 15:51:57.177348 23563 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:57.177659 23568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.142774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:45983 (local address 127.15.196.195:60351)
0929 15:51:57.143796 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:57.143841 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:57.146414 (+ 2573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:57.147179 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:57.147184 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:57.148252 (+ 1068us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:51:57.149206 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:57.149232 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.150760 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.150765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:57.152864 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:57.152876 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.153900 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.153906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:57.154246 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.154959 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:51:57.154982 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:51:57.159076 (+ 4094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:51:57.163876 (+ 4800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:57.163886 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:57.163905 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:57.164593 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:57.165079 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:51:57.165083 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:51:57.165087 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:51:57.165388 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:51:57.166061 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:51:57.166065 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:51:57.166656 (+ 591us) client_negotiation.cc:769] Sending connection context
0929 15:51:57.168102 (+ 1446us) client_negotiation.cc:240] Negotiation successful
0929 15:51:57.174625 (+ 6523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":6628,"spinlock_wait_cycles":46336,"thread_start_us":500,"threads_started":1}
I20260929 15:51:57.178012 23563 heartbeater.cc:511] Master 127.15.196.254:44195 requested a full tablet report, sending...
I20260929 15:51:57.178586 23564 heartbeater.cc:347] Connected to a master server at 127.15.196.253:45983
I20260929 15:51:57.178710 23564 heartbeater.cc:464] Registering TS with master...
I20260929 15:51:57.179068 22306 ts_manager.cc:194] Re-registered known tserver with Master: d2e4767290964f7fa24e766c3c1248b7 (127.15.196.195:33907)
I20260929 15:51:57.179157 23564 heartbeater.cc:511] Master 127.15.196.253:45983 requested a full tablet report, sending...
I20260929 15:51:57.180198 22306 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:59745
I20260929 15:51:57.181079 22904 ts_manager.cc:194] Registered new tserver with Master: d2e4767290964f7fa24e766c3c1248b7 (127.15.196.195:33907)
I20260929 15:51:57.202143 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.154326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48552 (local address 127.15.196.253:45983)
0929 15:51:57.168064 (+ 13738us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:57.168069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:57.184304 (+ 16235us) server_negotiation.cc:407] Connection header received
0929 15:51:57.184529 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:57.184533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:57.184632 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:57.184727 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:57.187585 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.188735 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.190243 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.190937 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.191032 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.195547 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:57.195555 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:57.195562 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:57.195607 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:57.199126 (+ 3519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.200088 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.200093 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.200096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.200242 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.200636 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.200645 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.200648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.200952 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:57.201224 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:57.201565 (+ 341us) server_negotiation.cc:299] Negotiation successful
0929 15:51:57.201775 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13629,"thread_start_us":32642,"threads_started":1}
I20260929 15:51:57.202345 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.185134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41832 (local address 127.15.196.252:34407)
0929 15:51:57.185391 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:57.185399 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:57.186478 (+ 1079us) server_negotiation.cc:407] Connection header received
0929 15:51:57.186668 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:57.186676 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:57.186828 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:57.186946 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:57.189019 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.190091 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.191368 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.191782 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.191935 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.195530 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:57.195536 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:57.195539 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:57.195582 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:57.198888 (+ 3306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.199956 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.199962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.199965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.200129 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.200582 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.200589 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.200594 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.200979 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:57.201217 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:57.201795 (+ 578us) server_negotiation.cc:299] Negotiation successful
0929 15:51:57.201998 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260929 15:51:57.203181 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.160355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51220 (local address 127.15.196.254:44195)
0929 15:51:57.175127 (+ 14772us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:57.175138 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:57.186390 (+ 11252us) server_negotiation.cc:407] Connection header received
0929 15:51:57.187424 (+ 1034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:57.187429 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:57.187539 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:57.187636 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:57.190210 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.191248 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.192957 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.193391 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.193535 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.196337 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:57.196346 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:57.196352 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:57.196396 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:57.199271 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.200134 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.200139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.200143 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.200293 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.200779 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.200784 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.200787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.201146 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:57.201357 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:57.202601 (+ 1244us) server_negotiation.cc:299] Negotiation successful
0929 15:51:57.202830 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14672,"thread_start_us":25053,"threads_started":1}
I20260929 15:51:57.206866 22306 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:51:57.214334 22306 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:51:57.256582 23592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.234286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:33961 (local address 127.0.0.1:36298)
0929 15:51:57.248298 (+ 14012us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:57.248313 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:57.248460 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:57.249053 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:57.249058 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:57.249082 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:57.249514 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:57.249543 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.251274 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.251278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:57.253630 (+ 2352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:57.253649 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.255747 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.255755 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:57.255952 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.255989 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 15:51:57.256085 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 15:51:57.256217 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13864,"thread_start_us":13674,"threads_started":1}
I20260929 15:51:57.257368 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.234859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36298 (local address 127.15.196.194:33961)
0929 15:51:57.247747 (+ 12888us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:57.247760 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:57.248418 (+ 658us) server_negotiation.cc:407] Connection header received
0929 15:51:57.248629 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:57.248636 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:57.248862 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:57.249075 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:57.249671 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.251115 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.253815 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.255252 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.255433 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.255624 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:57.256866 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0929 15:51:57.257096 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12725,"thread_start_us":12623,"threads_started":1}
I20260929 15:51:57.260360 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.232620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:37331 (local address 127.0.0.1:43740)
0929 15:51:57.245424 (+ 12804us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:57.245475 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:57.245624 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:57.251586 (+ 5962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:57.251591 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:57.251621 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:57.251999 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:57.252012 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.253523 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.253529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:57.255643 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:57.255659 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.259559 (+ 3900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.259565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:57.259760 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.259794 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 15:51:57.259885 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 15:51:57.260024 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12627,"thread_start_us":13107,"threads_started":1}
I20260929 15:51:57.261288 23356 tablet_service.cc:1467] Processing CreateTablet for tablet dfdedd41c12445139066dcb9180d36dd (DEFAULT_TABLE table=test-table [id=46ce2c0dbee14c0fa5dbb442d5ef6d77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:57.262753 23356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfdedd41c12445139066dcb9180d36dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:57.263109 23588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.232062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:33907 (local address 127.0.0.1:51416)
0929 15:51:57.232869 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:57.232894 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:57.233061 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:57.253272 (+ 20211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:57.253277 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:57.253305 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:57.255430 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:57.255444 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.258767 (+ 3323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.258771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:57.260783 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:57.260797 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.262326 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.262330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:57.262456 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.262478 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 15:51:57.262623 (+ 145us) client_negotiation.cc:240] Negotiation successful
0929 15:51:57.262758 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":330,"threads_started":1}
I20260929 15:51:57.263453 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.232480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51416 (local address 127.15.196.195:33907)
0929 15:51:57.252692 (+ 20212us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:57.252700 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:57.252732 (+ 32us) server_negotiation.cc:407] Connection header received
0929 15:51:57.252820 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:57.252828 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:57.253061 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:57.253352 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:57.255662 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.258603 (+ 2941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.260947 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.261876 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.262084 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.262197 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:57.262639 (+ 442us) server_negotiation.cc:299] Negotiation successful
0929 15:51:57.262879 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20030,"thread_start_us":19852,"threads_started":1}
I20260929 15:51:57.267059 23497 tablet_service.cc:1467] Processing CreateTablet for tablet dfdedd41c12445139066dcb9180d36dd (DEFAULT_TABLE table=test-table [id=46ce2c0dbee14c0fa5dbb442d5ef6d77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:57.268036 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.232842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43740 (local address 127.15.196.193:37331)
0929 15:51:57.250811 (+ 17969us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:57.250823 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:57.250862 (+ 39us) server_negotiation.cc:407] Connection header received
0929 15:51:57.251004 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:57.251017 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:57.251363 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:57.251615 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:57.252228 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.253384 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.255852 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.257720 (+ 1868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.257877 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.258054 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:57.267468 (+ 9414us) server_negotiation.cc:299] Negotiation successful
0929 15:51:57.267734 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17744,"thread_start_us":18484,"threads_started":1}
I20260929 15:51:57.268508 23497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfdedd41c12445139066dcb9180d36dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:57.271880 23208 tablet_service.cc:1467] Processing CreateTablet for tablet dfdedd41c12445139066dcb9180d36dd (DEFAULT_TABLE table=test-table [id=46ce2c0dbee14c0fa5dbb442d5ef6d77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:57.273208 23208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfdedd41c12445139066dcb9180d36dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:57.284444 23594 tablet_bootstrap.cc:492] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f: Bootstrap starting.
I20260929 15:51:57.285003 23595 tablet_bootstrap.cc:492] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7: Bootstrap starting.
I20260929 15:51:57.288105 23594 tablet_bootstrap.cc:654] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:57.288493 23595 tablet_bootstrap.cc:654] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:57.289371 23596 tablet_bootstrap.cc:492] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008: Bootstrap starting.
I20260929 15:51:57.289474 23594 log.cc:824] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:57.289649 23595 log.cc:824] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:57.292856 23596 tablet_bootstrap.cc:654] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:57.293133 23594 tablet_bootstrap.cc:492] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f: No bootstrap required, opened a new log
I20260929 15:51:57.293407 23594 ts_tablet_manager.cc:1397] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 15:51:57.293582 23595 tablet_bootstrap.cc:492] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7: No bootstrap required, opened a new log
I20260929 15:51:57.293830 23595 ts_tablet_manager.cc:1397] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 15:51:57.294070 23596 log.cc:824] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008: Log is configured to *not* fsync() on all Append() calls
I20260929 15:51:57.297828 23596 tablet_bootstrap.cc:492] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008: No bootstrap required, opened a new log
I20260929 15:51:57.298094 23596 ts_tablet_manager.cc:1397] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260929 15:51:57.304127 23594 raft_consensus.cc:348] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } } peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } }
I20260929 15:51:57.304560 23594 raft_consensus.cc:374] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:57.304665 23594 raft_consensus.cc:729] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cf3ae47c7cb4f09a336e751fa90261f, State: Initialized, Role: FOLLOWER
I20260929 15:51:57.305204 23595 raft_consensus.cc:348] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } } peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } }
I20260929 15:51:57.305306 23596 raft_consensus.cc:348] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } } peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } }
I20260929 15:51:57.305431 23594 consensus_queue.cc:260] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } } peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } }
I20260929 15:51:57.305630 23595 raft_consensus.cc:374] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:57.305655 23596 raft_consensus.cc:374] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:57.305717 23595 raft_consensus.cc:729] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e4767290964f7fa24e766c3c1248b7, State: Initialized, Role: FOLLOWER
I20260929 15:51:57.305750 23596 raft_consensus.cc:729] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94640710d80f4186bd98dcc97d749008, State: Initialized, Role: FOLLOWER
I20260929 15:51:57.306347 23596 consensus_queue.cc:260] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } } peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } }
I20260929 15:51:57.306393 23595 consensus_queue.cc:260] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } } peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } }
I20260929 15:51:57.312151 23595 ts_tablet_manager.cc:1428] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
I20260929 15:51:57.312448 23563 heartbeater.cc:503] Master 127.15.196.254:44195 was elected leader, sending a full tablet report...
I20260929 15:51:57.315788 23594 ts_tablet_manager.cc:1428] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f: Time spent starting tablet: real 0.022s user 0.018s sys 0.004s
I20260929 15:51:57.316454 23422 heartbeater.cc:503] Master 127.15.196.254:44195 was elected leader, sending a full tablet report...
I20260929 15:51:57.323014 23596 ts_tablet_manager.cc:1428] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008: Time spent starting tablet: real 0.025s user 0.021s sys 0.005s
W20260929 15:51:57.393035 23565 tablet.cc:2369] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 15:51:57.408663 23276 tablet.cc:2369] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:51:57.478964 23602 raft_consensus.cc:482] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:51:57.479406 23602 raft_consensus.cc:504] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } } peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } }
I20260929 15:51:57.481549 23602 leader_election.cc:288] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94640710d80f4186bd98dcc97d749008 (127.15.196.193:37331), 5cf3ae47c7cb4f09a336e751fa90261f (127.15.196.194:33961)
I20260929 15:51:57.489822 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.482482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:51499 (local address 127.15.196.194:33961)
0929 15:51:57.482697 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:57.482703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:57.482722 (+ 19us) server_negotiation.cc:407] Connection header received
0929 15:51:57.482790 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:57.482794 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:57.482863 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:57.482949 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:57.483461 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.484340 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.487760 (+ 3420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.488678 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.488871 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.488958 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:57.489416 (+ 458us) server_negotiation.cc:299] Negotiation successful
0929 15:51:57.489562 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 15:51:57.489880 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.481855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:33961 (local address 127.15.196.195:51499)
0929 15:51:57.482219 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:57.482235 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:57.482343 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:57.482997 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:57.483004 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:57.483027 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:57.483269 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:57.483279 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.484501 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.484510 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:57.487531 (+ 3021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:57.487550 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.488896 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.488903 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:57.489158 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.489241 (+ 83us) client_negotiation.cc:769] Sending connection context
0929 15:51:57.489369 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 15:51:57.489539 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":296}
I20260929 15:51:57.490212 23568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.481654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:37331 (local address 127.15.196.195:32995)
0929 15:51:57.481882 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:57.481908 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:57.482049 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:57.482812 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:57.482817 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:57.482845 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:57.483112 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:57.483124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.484577 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.484582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:57.487531 (+ 2949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:57.487550 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.489181 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.489189 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:57.489477 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.489511 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 15:51:57.489605 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 15:51:57.489741 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":305}
I20260929 15:51:57.491091 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.482284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:32995 (local address 127.15.196.193:37331)
0929 15:51:57.482489 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:57.482495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:57.482516 (+ 21us) server_negotiation.cc:407] Connection header received
0929 15:51:57.482606 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:57.482611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:57.482675 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:57.482790 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:57.483325 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.484416 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.487733 (+ 3317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.489020 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.489175 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.489257 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:57.490769 (+ 1512us) server_negotiation.cc:299] Negotiation successful
0929 15:51:57.490896 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 15:51:57.490984 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfdedd41c12445139066dcb9180d36dd" candidate_uuid: "d2e4767290964f7fa24e766c3c1248b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" is_pre_election: true
I20260929 15:51:57.491552 23376 raft_consensus.cc:2454] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e4767290964f7fa24e766c3c1248b7 in term 0.
I20260929 15:51:57.492147 23228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfdedd41c12445139066dcb9180d36dd" candidate_uuid: "d2e4767290964f7fa24e766c3c1248b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94640710d80f4186bd98dcc97d749008" is_pre_election: true
I20260929 15:51:57.492277 23450 leader_election.cc:302] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cf3ae47c7cb4f09a336e751fa90261f, d2e4767290964f7fa24e766c3c1248b7; no voters:
I20260929 15:51:57.492704 23602 raft_consensus.cc:2781] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:51:57.492697 23228 raft_consensus.cc:2454] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e4767290964f7fa24e766c3c1248b7 in term 0.
I20260929 15:51:57.492941 23602 raft_consensus.cc:482] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:51:57.493070 23602 raft_consensus.cc:3037] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:57.495669 23602 raft_consensus.cc:504] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } } peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } }
I20260929 15:51:57.496323 23602 leader_election.cc:288] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [CANDIDATE]: Term 1 election: Requested vote from peers 94640710d80f4186bd98dcc97d749008 (127.15.196.193:37331), 5cf3ae47c7cb4f09a336e751fa90261f (127.15.196.194:33961)
I20260929 15:51:57.496757 23228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfdedd41c12445139066dcb9180d36dd" candidate_uuid: "d2e4767290964f7fa24e766c3c1248b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94640710d80f4186bd98dcc97d749008"
I20260929 15:51:57.496874 23376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfdedd41c12445139066dcb9180d36dd" candidate_uuid: "d2e4767290964f7fa24e766c3c1248b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cf3ae47c7cb4f09a336e751fa90261f"
I20260929 15:51:57.496963 23228 raft_consensus.cc:3037] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:57.497133 23376 raft_consensus.cc:3037] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:57.499413 23228 raft_consensus.cc:2454] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e4767290964f7fa24e766c3c1248b7 in term 1.
I20260929 15:51:57.499859 23453 leader_election.cc:302] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94640710d80f4186bd98dcc97d749008, d2e4767290964f7fa24e766c3c1248b7; no voters:
I20260929 15:51:57.500174 23376 raft_consensus.cc:2454] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e4767290964f7fa24e766c3c1248b7 in term 1.
I20260929 15:51:57.500192 23602 raft_consensus.cc:2781] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:57.500733 23602 raft_consensus.cc:686] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [term 1 LEADER]: Becoming Leader. State: Replica: d2e4767290964f7fa24e766c3c1248b7, State: Running, Role: LEADER
I20260929 15:51:57.501272 23602 consensus_queue.cc:237] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } } peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } }
I20260929 15:51:57.506644 22306 catalog_manager.cc:5495] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2e4767290964f7fa24e766c3c1248b7 (127.15.196.195). New cstate: current_term: 1 leader_uuid: "d2e4767290964f7fa24e766c3c1248b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } health_report { overall_health: UNKNOWN } } }
I20260929 15:51:57.541189 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.535224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51428 (local address 127.15.196.195:33907)
0929 15:51:57.535422 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:57.535427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:57.535444 (+ 17us) server_negotiation.cc:407] Connection header received
0929 15:51:57.535565 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:57.535569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:57.535643 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:57.535762 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:51:57.536164 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.536868 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.538823 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.539234 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.539411 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.539913 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:51:57.540430 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:51:57.540613 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:57.540819 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 15:51:57.540907 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 15:51:57.552098 23228 raft_consensus.cc:1261] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008 [term 1 FOLLOWER]: Refusing update from remote peer d2e4767290964f7fa24e766c3c1248b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:57.552084 23376 raft_consensus.cc:1261] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f [term 1 FOLLOWER]: Refusing update from remote peer d2e4767290964f7fa24e766c3c1248b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:57.553241 23602 consensus_queue.cc:1035] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:57.553576 23603 consensus_queue.cc:1035] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 15:51:57.557612 23424 tablet.cc:2369] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:51:57.567408 23609 mvcc.cc:205] Tried to move back new op lower bound from 7334695393482088448 to 7334695393289273344. Current Snapshot: MvccSnapshot[applied={T|T < 7334695393482088448}]
I20260929 15:51:57.572152 23608 mvcc.cc:205] Tried to move back new op lower bound from 7334695393482088448 to 7334695393289273344. Current Snapshot: MvccSnapshot[applied={T|T < 7334695393482088448}]
I20260929 15:51:57.589268 22306 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:57.589536 22306 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:57.596055 22306 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a54661c127145c58a343e416a1c63da: Sending DeleteTablet for 3 replicas of tablet dfdedd41c12445139066dcb9180d36dd
I20260929 15:51:57.597213 23497 tablet_service.cc:1514] Processing DeleteTablet for tablet dfdedd41c12445139066dcb9180d36dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:57 UTC) from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:51416
I20260929 15:51:57.597714 23208 tablet_service.cc:1514] Processing DeleteTablet for tablet dfdedd41c12445139066dcb9180d36dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:57 UTC) from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:43740
I20260929 15:51:57.597982 23356 tablet_service.cc:1514] Processing DeleteTablet for tablet dfdedd41c12445139066dcb9180d36dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:57 UTC) from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:36298
I20260929 15:51:57.598380 23616 tablet_replica.cc:330] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7: stopping tablet replica
I20260929 15:51:57.598953 23616 raft_consensus.cc:2229] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:51:57.599915 23616 raft_consensus.cc:2258] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:57.602437 23616 ts_tablet_manager.cc:1905] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:57.610265 23618 tablet_replica.cc:330] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f: stopping tablet replica
I20260929 15:51:57.610743 23618 raft_consensus.cc:2229] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:57.611263 23618 raft_consensus.cc:2258] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:57.612351 23616 ts_tablet_manager.cc:1918] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:51:57.612563 23616 log.cc:1196] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/dfdedd41c12445139066dcb9180d36dd
I20260929 15:51:57.612731 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 22274
I20260929 15:51:57.613317 23616 ts_tablet_manager.cc:1939] T dfdedd41c12445139066dcb9180d36dd P d2e4767290964f7fa24e766c3c1248b7: Deleting consensus metadata
I20260929 15:51:57.613637 23618 ts_tablet_manager.cc:1905] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:57.614117 22336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:57.617592 23617 tablet_replica.cc:330] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008: stopping tablet replica
I20260929 15:51:57.618988 23617 raft_consensus.cc:2229] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:57.619611 23617 raft_consensus.cc:2258] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:57.623059 23618 ts_tablet_manager.cc:1918] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:51:57.623234 23618 log.cc:1196] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/dfdedd41c12445139066dcb9180d36dd
I20260929 15:51:57.624579 23618 ts_tablet_manager.cc:1939] T dfdedd41c12445139066dcb9180d36dd P 5cf3ae47c7cb4f09a336e751fa90261f: Deleting consensus metadata
I20260929 15:51:57.626672 23617 ts_tablet_manager.cc:1905] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:57.634351 23617 ts_tablet_manager.cc:1918] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:51:57.634542 23617 log.cc:1196] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/dfdedd41c12445139066dcb9180d36dd
I20260929 15:51:57.635131 23617 ts_tablet_manager.cc:1939] T dfdedd41c12445139066dcb9180d36dd P 94640710d80f4186bd98dcc97d749008: Deleting consensus metadata
I20260929 15:51:57.645977 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.600010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48562 (local address 127.15.196.253:45983)
0929 15:51:57.600250 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:57.600259 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:57.610892 (+ 10633us) server_negotiation.cc:407] Connection header received
0929 15:51:57.614250 (+ 3358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:57.614261 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:57.614418 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:57.614518 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:57.620055 (+ 5537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.621336 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.623806 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.624495 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.624608 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.638613 (+ 14005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:57.638622 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:57.638628 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:57.638678 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:57.642544 (+ 3866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.643488 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.643497 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.643502 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.643679 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.644118 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.644124 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.644128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.644541 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:57.644815 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:57.645318 (+ 503us) server_negotiation.cc:299] Negotiation successful
0929 15:51:57.645570 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 15:51:57.663394 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.611559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41844 (local address 127.15.196.252:34407)
0929 15:51:57.611722 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:57.611728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:57.645870 (+ 34142us) server_negotiation.cc:407] Connection header received
0929 15:51:57.646375 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:57.646382 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:57.646465 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:57.646545 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:57.648451 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.649760 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.651758 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.652477 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.652625 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.656011 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:57.656019 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:57.656026 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:57.656105 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:57.660274 (+ 4169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.661062 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.661070 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.661074 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.661233 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.661660 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.661666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.661670 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.662076 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:57.662385 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:57.662671 (+ 286us) server_negotiation.cc:299] Negotiation successful
0929 15:51:57.662904 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 15:51:57.764134 22290 catalog_manager.cc:4841] TS d2e4767290964f7fa24e766c3c1248b7 (127.15.196.195:33907): tablet dfdedd41c12445139066dcb9180d36dd (table test-table [id=46ce2c0dbee14c0fa5dbb442d5ef6d77]) successfully deleted
W20260929 15:51:57.764506 22290 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.15.196.195:33907, user_credentials={real_user=slave}} blocked reactor thread for 142189us
I20260929 15:51:57.765008 22290 catalog_manager.cc:4841] TS 5cf3ae47c7cb4f09a336e751fa90261f (127.15.196.194:33961): tablet dfdedd41c12445139066dcb9180d36dd (table test-table [id=46ce2c0dbee14c0fa5dbb442d5ef6d77]) successfully deleted
I20260929 15:51:57.765444 22293 catalog_manager.cc:4841] TS 94640710d80f4186bd98dcc97d749008 (127.15.196.193:37331): tablet dfdedd41c12445139066dcb9180d36dd (table test-table [id=46ce2c0dbee14c0fa5dbb442d5ef6d77]) successfully deleted
I20260929 15:51:57.766021 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 22274
I20260929 15:51:57.810560 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.791738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48578 (local address 127.15.196.253:45983)
0929 15:51:57.791966 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:57.791972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:57.792992 (+ 1020us) server_negotiation.cc:407] Connection header received
0929 15:51:57.793170 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:57.793179 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:57.793289 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:57.793394 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:57.795669 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.796793 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.798907 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.799582 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.799705 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.803823 (+ 4118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:57.803830 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:57.803838 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:57.803877 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:57.807445 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.808387 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.808392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.808395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.808561 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.809049 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.809054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.809057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.809338 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:57.809603 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:57.809933 (+ 330us) server_negotiation.cc:299] Negotiation successful
0929 15:51:57.810155 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 15:51:57.811792 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:57.793842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41850 (local address 127.15.196.252:34407)
0929 15:51:57.794055 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:57.794059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:57.795650 (+ 1591us) server_negotiation.cc:407] Connection header received
0929 15:51:57.795902 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:57.795911 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:57.796006 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:57.796114 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:57.797893 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.798595 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.800345 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:57.800944 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:57.801096 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:57.803655 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:57.803664 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:57.803668 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:57.803735 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:57.807233 (+ 3498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.808216 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.808222 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.808226 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.808512 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:57.809032 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:57.809039 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:57.809043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:57.809474 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:57.809718 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:57.811154 (+ 1436us) server_negotiation.cc:299] Negotiation successful
0929 15:51:57.811433 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 15:51:58.576998 23627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:58.576309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.193:52697)
0929 15:51:58.576726 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:58.576837 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":198,"threads_started":1}
W20260929 15:51:58.577425 23274 heartbeater.cc:644] Failed to heartbeat to 127.15.196.254:44195 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
I20260929 15:51:58.585387 23628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:58.584422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.194:45091)
0929 15:51:58.584941 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:58.585133 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":257,"threads_started":1}
I20260929 15:51:58.585851 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:58.584836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.195:42361)
0929 15:51:58.585533 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:58.585660 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":488,"threads_started":1}
W20260929 15:51:58.585947 23422 heartbeater.cc:644] Failed to heartbeat to 127.15.196.254:44195 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
W20260929 15:51:58.586279 23563 heartbeater.cc:644] Failed to heartbeat to 127.15.196.254:44195 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
I20260929 15:51:59.096361 23630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a54661c127145c58a343e416a1c63da)
I20260929 15:51:59.096597 23630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } }
I20260929 15:51:59.097250 23631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4a54661c127145c58a343e416a1c63da)
I20260929 15:51:59.097488 23631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } }
I20260929 15:51:59.098155 23630 leader_election.cc:288] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a54661c127145c58a343e416a1c63da (127.15.196.254:44195), 1cb3483982a74bf5b724b88fee7f15e3 (127.15.196.252:34407)
I20260929 15:51:59.098829 23631 leader_election.cc:288] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a54661c127145c58a343e416a1c63da (127.15.196.254:44195), 5f752fb87cd64afc99afd30cd9da9cee (127.15.196.253:45983)
I20260929 15:51:59.100046 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.098265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.0.0.1:51620)
0929 15:51:59.099453 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.099700 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":973,"thread_start_us":800,"threads_started":1}
I20260929 15:51:59.101672 23634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.098870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.0.0.1:51634)
0929 15:51:59.101086 (+ 2216us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.101331 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2059,"threads_started":1}
W20260929 15:51:59.106612 22890 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
W20260929 15:51:59.106768 23079 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
I20260929 15:51:59.109609 23633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.108934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.0.0.1:51644)
0929 15:51:59.109195 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.109360 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260929 15:51:59.109859 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.109280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.0.0.1:51656)
0929 15:51:59.109473 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.109640 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 15:51:59.109830 23079 leader_election.cc:334] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4a54661c127145c58a343e416a1c63da (127.15.196.254:44195): Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
W20260929 15:51:59.110081 22890 leader_election.cc:334] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4a54661c127145c58a343e416a1c63da (127.15.196.254:44195): Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
I20260929 15:51:59.111675 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.101074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41858 (local address 127.15.196.252:34407)
0929 15:51:59.102635 (+ 1561us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:59.102644 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:59.102666 (+ 22us) server_negotiation.cc:407] Connection header received
0929 15:51:59.102739 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:59.102747 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:59.102865 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:59.103716 (+ 851us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:59.104035 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.105600 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.109565 (+ 3965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.110692 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.110829 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.110879 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:59.111224 (+ 345us) server_negotiation.cc:299] Negotiation successful
0929 15:51:59.111405 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1400,"thread_start_us":1233,"threads_started":1}
I20260929 15:51:59.111680 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.099440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:34407 (local address 127.0.0.1:41858)
0929 15:51:59.100285 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.100308 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:59.100471 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:59.103068 (+ 2597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:59.103076 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:59.103116 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:59.103542 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:59.103558 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.106665 (+ 3107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.106671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:59.109385 (+ 2714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:59.109401 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.110862 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.110868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:59.111051 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.111087 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 15:51:59.111198 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 15:51:59.111363 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"mutex_wait_us":1,"thread_start_us":6736,"threads_started":1}
I20260929 15:51:59.112433 23104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f752fb87cd64afc99afd30cd9da9cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1cb3483982a74bf5b724b88fee7f15e3" is_pre_election: true
I20260929 15:51:59.112782 23104 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f752fb87cd64afc99afd30cd9da9cee in term 1.
I20260929 15:51:59.113322 22890 leader_election.cc:302] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cb3483982a74bf5b724b88fee7f15e3, 5f752fb87cd64afc99afd30cd9da9cee; no voters: 4a54661c127145c58a343e416a1c63da
I20260929 15:51:59.113690 23630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 15:51:59.113837 23630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4a54661c127145c58a343e416a1c63da)
I20260929 15:51:59.113962 23630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 1 FOLLOWER]: Advancing to term 2
I20260929 15:51:59.114852 23634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.099113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:45983 (local address 127.0.0.1:48588)
0929 15:51:59.101873 (+ 2760us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.101894 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:59.102032 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:59.106628 (+ 4596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:59.106633 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:59.106659 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:59.106994 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:59.107008 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.110554 (+ 3546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.110560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:59.112501 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:59.112512 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.114052 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.114059 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:59.114228 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.114250 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 15:51:59.114356 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 15:51:59.114507 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2654,"mutex_wait_us":25,"thread_start_us":1653,"threads_started":1}
I20260929 15:51:59.114907 23636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.099440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48588 (local address 127.15.196.253:45983)
0929 15:51:59.106269 (+ 6829us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:59.106283 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:59.106308 (+ 25us) server_negotiation.cc:407] Connection header received
0929 15:51:59.106383 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:59.106391 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:59.106463 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:59.107137 (+ 674us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:59.107433 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.110413 (+ 2980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.112643 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.113872 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.114072 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.114143 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:59.114390 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 15:51:59.114520 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":737,"server-negotiator.queue_time_us":6646,"thread_start_us":6429,"threads_started":1}
I20260929 15:51:59.115561 22914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1cb3483982a74bf5b724b88fee7f15e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5f752fb87cd64afc99afd30cd9da9cee" is_pre_election: true
I20260929 15:51:59.116729 23630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } }
I20260929 15:51:59.117141 22914 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1cb3483982a74bf5b724b88fee7f15e3 in current term 2: Already voted for candidate 5f752fb87cd64afc99afd30cd9da9cee in this term.
I20260929 15:51:59.117383 23630 leader_election.cc:288] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [CANDIDATE]: Term 2 election: Requested vote from peers 4a54661c127145c58a343e416a1c63da (127.15.196.254:44195), 1cb3483982a74bf5b724b88fee7f15e3 (127.15.196.252:34407)
I20260929 15:51:59.117976 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.117549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.0.0.1:51666)
0929 15:51:59.117701 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.117793 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260929 15:51:59.118247 23077 leader_election.cc:302] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cb3483982a74bf5b724b88fee7f15e3; no voters: 4a54661c127145c58a343e416a1c63da, 5f752fb87cd64afc99afd30cd9da9cee
I20260929 15:51:59.118609 23631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 1 FOLLOWER]: Advancing to term 2
I20260929 15:51:59.118871 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.118571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.0.0.1:51680)
0929 15:51:59.118725 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.118784 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260929 15:51:59.119050 22890 leader_election.cc:334] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4a54661c127145c58a343e416a1c63da (127.15.196.254:44195): Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
I20260929 15:51:59.119097 23104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f752fb87cd64afc99afd30cd9da9cee" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1cb3483982a74bf5b724b88fee7f15e3"
I20260929 15:51:59.120795 23631 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 15:51:59.122848 23104 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f752fb87cd64afc99afd30cd9da9cee in term 2.
I20260929 15:51:59.123602 22890 leader_election.cc:302] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1cb3483982a74bf5b724b88fee7f15e3, 5f752fb87cd64afc99afd30cd9da9cee; no voters: 4a54661c127145c58a343e416a1c63da
I20260929 15:51:59.123907 23630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 2 FOLLOWER]: Leader election won for term 2
I20260929 15:51:59.124377 23630 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [term 2 LEADER]: Becoming Leader. State: Replica: 5f752fb87cd64afc99afd30cd9da9cee, State: Running, Role: LEADER
I20260929 15:51:59.124752 23630 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } }
I20260929 15:51:59.126590 23640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f752fb87cd64afc99afd30cd9da9cee. Latest consensus state: current_term: 2 leader_uuid: "5f752fb87cd64afc99afd30cd9da9cee" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } } }
I20260929 15:51:59.127031 23640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:59.127682 23642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:51:59.131608 23642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:51:59.132406 23642 catalog_manager.cc:1259] Loaded cluster ID: 894250600ca246a284b71fbfe3c54beb
I20260929 15:51:59.132498 23642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:51:59.133658 23642 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:51:59.134441 23642 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: Loaded TSK: 0
I20260929 15:51:59.135337 23642 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:51:59.194298 22903 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:51:59.201825 22903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:51:59.206334 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.205885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.0.0.1:51708)
0929 15:51:59.206101 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.206186 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260929 15:51:59.206382 23104 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [term 2 FOLLOWER]: Refusing update from remote peer 5f752fb87cd64afc99afd30cd9da9cee: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260929 15:51:59.207351 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.206991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.0.0.1:51722)
0929 15:51:59.207122 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.207190 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 15:51:59.207460 23640 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260929 15:51:59.207741 22890 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee -> Peer 4a54661c127145c58a343e416a1c63da (127.15.196.254:44195): Couldn't send request to peer 4a54661c127145c58a343e416a1c63da. Status: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 15:51:59.210278 23631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f752fb87cd64afc99afd30cd9da9cee. Latest consensus state: current_term: 2 leader_uuid: "5f752fb87cd64afc99afd30cd9da9cee" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } } }
I20260929 15:51:59.210601 23631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:51:59.211963 23630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "5f752fb87cd64afc99afd30cd9da9cee" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } } }
I20260929 15:51:59.212275 23630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee [sys.catalog]: This master's current role is: LEADER
I20260929 15:51:59.212632 23644 mvcc.cc:205] Tried to move back new op lower bound from 7334695400261890048 to 7334695399938256896. Current Snapshot: MvccSnapshot[applied={T|T < 7334695400261890048}]
I20260929 15:51:59.213150 23631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "5f752fb87cd64afc99afd30cd9da9cee" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4a54661c127145c58a343e416a1c63da" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44195 } } peers { permanent_uuid: "5f752fb87cd64afc99afd30cd9da9cee" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 45983 } attrs { promote: false } } peers { permanent_uuid: "1cb3483982a74bf5b724b88fee7f15e3" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 34407 } attrs { promote: false } } }
I20260929 15:51:59.213387 23631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb3483982a74bf5b724b88fee7f15e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:51:59.229975 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.222936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:33907 (local address 127.0.0.1:51436)
0929 15:51:59.223070 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.223086 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:59.223182 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:59.223915 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:59.223919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:59.223941 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:59.224201 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:59.224209 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.225511 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.225516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:59.227577 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:59.227590 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.229254 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.229263 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:59.229449 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.229477 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 15:51:59.229530 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 15:51:59.229636 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 15:51:59.229928 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.221592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:33961 (local address 127.0.0.1:36310)
0929 15:51:59.221788 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.221810 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:59.221929 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:59.222983 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:59.222989 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:59.223015 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:59.223239 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:59.223258 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.224920 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.224926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:59.227332 (+ 2406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:59.227347 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.229026 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.229032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:59.229281 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.229327 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 15:51:59.229397 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 15:51:59.229556 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":57}
I20260929 15:51:59.230765 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.222152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36310 (local address 127.15.196.194:33961)
0929 15:51:59.222665 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:59.222669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:59.222684 (+ 15us) server_negotiation.cc:407] Connection header received
0929 15:51:59.222742 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:59.222745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:59.222800 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:59.223444 (+ 644us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:59.223650 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.224766 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.227535 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.228844 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.230169 (+ 1325us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.230248 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:59.230302 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 15:51:59.230456 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":302,"threads_started":1}
I20260929 15:51:59.231791 23356 tablet_service.cc:1467] Processing CreateTablet for tablet b77c1d24ce9a46388cd1baece508bef5 (DEFAULT_TABLE table=test-table [id=67cd89da02524b65a235bcec1a0d2321]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:59.232076 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.223106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51436 (local address 127.15.196.195:33907)
0929 15:51:59.223609 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:59.223616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:59.223634 (+ 18us) server_negotiation.cc:407] Connection header received
0929 15:51:59.223725 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:59.223729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:59.223804 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:59.223901 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:59.224356 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.225365 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.227785 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.229011 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.231451 (+ 2440us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.231545 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:59.231593 (+ 48us) server_negotiation.cc:299] Negotiation successful
0929 15:51:59.231725 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":265,"threads_started":1}
I20260929 15:51:59.232300 23356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b77c1d24ce9a46388cd1baece508bef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:59.232734 23497 tablet_service.cc:1467] Processing CreateTablet for tablet b77c1d24ce9a46388cd1baece508bef5 (DEFAULT_TABLE table=test-table [id=67cd89da02524b65a235bcec1a0d2321]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:59.233134 23497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b77c1d24ce9a46388cd1baece508bef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:59.238492 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.223801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:37331 (local address 127.0.0.1:43756)
0929 15:51:59.230211 (+ 6410us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.230230 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:59.230337 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:59.230768 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:59.230776 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:59.230810 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:59.231082 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:59.231096 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.234688 (+ 3592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.234696 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:59.236473 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:59.236487 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.237682 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.237688 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:59.237880 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.237904 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 15:51:59.238017 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 15:51:59.238178 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6322,"thread_start_us":11813,"threads_started":1}
I20260929 15:51:59.238848 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.224435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43756 (local address 127.15.196.193:37331)
0929 15:51:59.224909 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:59.224915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:59.230334 (+ 5419us) server_negotiation.cc:407] Connection header received
0929 15:51:59.230463 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:59.230472 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:59.230589 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:59.231330 (+ 741us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:59.231545 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.232989 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.236671 (+ 3682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.237453 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.237696 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.237777 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:59.238045 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 15:51:59.238186 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":231,"threads_started":1}
I20260929 15:51:59.239589 23208 tablet_service.cc:1467] Processing CreateTablet for tablet b77c1d24ce9a46388cd1baece508bef5 (DEFAULT_TABLE table=test-table [id=67cd89da02524b65a235bcec1a0d2321]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:51:59.240072 23208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b77c1d24ce9a46388cd1baece508bef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:51:59.244568 23652 tablet_bootstrap.cc:492] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f: Bootstrap starting.
I20260929 15:51:59.245823 23653 tablet_bootstrap.cc:492] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7: Bootstrap starting.
I20260929 15:51:59.246335 23654 tablet_bootstrap.cc:492] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008: Bootstrap starting.
I20260929 15:51:59.246695 23652 tablet_bootstrap.cc:654] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:59.248045 23653 tablet_bootstrap.cc:654] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:59.248196 23654 tablet_bootstrap.cc:654] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008: Neither blocks nor log segments found. Creating new log.
I20260929 15:51:59.250372 23653 tablet_bootstrap.cc:492] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7: No bootstrap required, opened a new log
I20260929 15:51:59.250521 23653 ts_tablet_manager.cc:1397] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 15:51:59.250574 23652 tablet_bootstrap.cc:492] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f: No bootstrap required, opened a new log
I20260929 15:51:59.250739 23652 ts_tablet_manager.cc:1397] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 15:51:59.251331 23654 tablet_bootstrap.cc:492] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008: No bootstrap required, opened a new log
I20260929 15:51:59.251487 23654 ts_tablet_manager.cc:1397] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 15:51:59.251437 23653 raft_consensus.cc:348] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } } peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } }
I20260929 15:51:59.251461 23652 raft_consensus.cc:348] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } } peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } }
I20260929 15:51:59.251690 23653 raft_consensus.cc:374] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:59.251734 23652 raft_consensus.cc:374] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:59.251791 23653 raft_consensus.cc:729] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e4767290964f7fa24e766c3c1248b7, State: Initialized, Role: FOLLOWER
I20260929 15:51:59.251837 23652 raft_consensus.cc:729] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cf3ae47c7cb4f09a336e751fa90261f, State: Initialized, Role: FOLLOWER
I20260929 15:51:59.252081 23653 consensus_queue.cc:260] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } } peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } }
I20260929 15:51:59.252095 23652 consensus_queue.cc:260] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } } peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } }
I20260929 15:51:59.252301 23654 raft_consensus.cc:348] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } } peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } }
I20260929 15:51:59.252591 23654 raft_consensus.cc:374] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:51:59.252717 23654 raft_consensus.cc:729] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94640710d80f4186bd98dcc97d749008, State: Initialized, Role: FOLLOWER
I20260929 15:51:59.253003 23654 consensus_queue.cc:260] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } } peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } }
I20260929 15:51:59.256866 23653 ts_tablet_manager.cc:1428] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260929 15:51:59.258001 23661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.256252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.195:59747)
0929 15:51:59.257521 (+ 1269us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.257737 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1132,"thread_start_us":993,"threads_started":1}
I20260929 15:51:59.260339 23564 heartbeater.cc:503] Master 127.15.196.253:45983 was elected leader, sending a full tablet report...
I20260929 15:51:59.261184 23652 ts_tablet_manager.cc:1428] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f: Time spent starting tablet: real 0.010s user 0.005s sys 0.003s
I20260929 15:51:59.261572 23661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.258695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.195:41653)
0929 15:51:59.261263 (+ 2568us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.261374 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2417}
I20260929 15:51:59.264617 23654 ts_tablet_manager.cc:1428] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008: Time spent starting tablet: real 0.013s user 0.002s sys 0.009s
I20260929 15:51:59.265743 23423 heartbeater.cc:503] Master 127.15.196.253:45983 was elected leader, sending a full tablet report...
I20260929 15:51:59.267055 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.265538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.193:44251)
0929 15:51:59.266379 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.266725 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":390,"threads_started":1}
I20260929 15:51:59.268585 23275 heartbeater.cc:503] Master 127.15.196.253:45983 was elected leader, sending a full tablet report...
I20260929 15:51:59.268647 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.267654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.193:54649)
0929 15:51:59.268186 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.268348 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":292,"threads_started":1}
I20260929 15:51:59.268751 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.262392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.194:55831)
0929 15:51:59.268367 (+ 5975us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.268554 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2612,"thread_start_us":2478,"threads_started":1}
I20260929 15:51:59.269696 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.269369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.194:47943)
0929 15:51:59.269523 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.269598 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260929 15:51:59.442958 23660 raft_consensus.cc:482] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:51:59.443187 23660 raft_consensus.cc:504] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } } peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } }
I20260929 15:51:59.444828 23660 leader_election.cc:288] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cf3ae47c7cb4f09a336e751fa90261f (127.15.196.194:33961), d2e4767290964f7fa24e766c3c1248b7 (127.15.196.195:33907)
I20260929 15:51:59.450941 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.445215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:48391 (local address 127.15.196.194:33961)
0929 15:51:59.445422 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:59.445429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:59.445453 (+ 24us) server_negotiation.cc:407] Connection header received
0929 15:51:59.445556 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:59.445562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:59.445638 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:59.445747 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:59.446430 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.447153 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.449173 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.449987 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.450162 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.450248 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:59.450530 (+ 282us) server_negotiation.cc:299] Negotiation successful
0929 15:51:59.450656 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 15:51:59.451004 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.445581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:40369 (local address 127.15.196.195:33907)
0929 15:51:59.445784 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:59.445789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:59.445811 (+ 22us) server_negotiation.cc:407] Connection header received
0929 15:51:59.445916 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:59.445921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:59.446002 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:59.446099 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:51:59.446611 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.447364 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.449306 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.450071 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.450250 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.450338 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:59.450607 (+ 269us) server_negotiation.cc:299] Negotiation successful
0929 15:51:59.450741 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 15:51:59.451025 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.445066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:33961 (local address 127.15.196.193:48391)
0929 15:51:59.445236 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.445258 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:59.445418 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:59.445908 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:59.445912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:59.445934 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:59.446265 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:59.446279 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.447313 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.447321 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:59.449011 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:59.449037 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.450162 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.450169 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:59.450361 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.450386 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 15:51:59.450488 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 15:51:59.450626 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 15:51:59.451035 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.445426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:33907 (local address 127.15.196.193:40369)
0929 15:51:59.445653 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.445671 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:51:59.445768 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:51:59.446143 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:51:59.446150 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:51:59.446172 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:51:59.446443 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:59.446450 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.447502 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.447508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:59.449119 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:51:59.449131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.450295 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.450302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:51:59.450451 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.450474 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 15:51:59.450580 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 15:51:59.450717 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":1}
I20260929 15:51:59.451697 23375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b77c1d24ce9a46388cd1baece508bef5" candidate_uuid: "94640710d80f4186bd98dcc97d749008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" is_pre_election: true
I20260929 15:51:59.452028 23375 raft_consensus.cc:2454] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94640710d80f4186bd98dcc97d749008 in term 0.
I20260929 15:51:59.452322 23517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b77c1d24ce9a46388cd1baece508bef5" candidate_uuid: "94640710d80f4186bd98dcc97d749008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e4767290964f7fa24e766c3c1248b7" is_pre_election: true
I20260929 15:51:59.452553 23159 leader_election.cc:302] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cf3ae47c7cb4f09a336e751fa90261f, 94640710d80f4186bd98dcc97d749008; no voters:
I20260929 15:51:59.452734 23517 raft_consensus.cc:2454] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94640710d80f4186bd98dcc97d749008 in term 0.
I20260929 15:51:59.452966 23660 raft_consensus.cc:2781] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:51:59.453130 23660 raft_consensus.cc:482] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:51:59.453258 23660 raft_consensus.cc:3037] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:59.455760 23660 raft_consensus.cc:504] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } } peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } }
I20260929 15:51:59.456290 23660 leader_election.cc:288] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [CANDIDATE]: Term 1 election: Requested vote from peers 5cf3ae47c7cb4f09a336e751fa90261f (127.15.196.194:33961), d2e4767290964f7fa24e766c3c1248b7 (127.15.196.195:33907)
I20260929 15:51:59.456780 23375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b77c1d24ce9a46388cd1baece508bef5" candidate_uuid: "94640710d80f4186bd98dcc97d749008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cf3ae47c7cb4f09a336e751fa90261f"
I20260929 15:51:59.456844 23517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b77c1d24ce9a46388cd1baece508bef5" candidate_uuid: "94640710d80f4186bd98dcc97d749008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e4767290964f7fa24e766c3c1248b7"
I20260929 15:51:59.456969 23375 raft_consensus.cc:3037] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:59.457099 23517 raft_consensus.cc:3037] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:51:59.459168 23375 raft_consensus.cc:2454] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94640710d80f4186bd98dcc97d749008 in term 1.
I20260929 15:51:59.459342 23517 raft_consensus.cc:2454] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94640710d80f4186bd98dcc97d749008 in term 1.
I20260929 15:51:59.459617 23159 leader_election.cc:302] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cf3ae47c7cb4f09a336e751fa90261f, 94640710d80f4186bd98dcc97d749008; no voters:
I20260929 15:51:59.459873 23660 raft_consensus.cc:2781] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:51:59.460402 23660 raft_consensus.cc:686] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [term 1 LEADER]: Becoming Leader. State: Replica: 94640710d80f4186bd98dcc97d749008, State: Running, Role: LEADER
I20260929 15:51:59.461244 23660 consensus_queue.cc:237] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } } peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } }
I20260929 15:51:59.461467 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.460959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.193:49219)
0929 15:51:59.461125 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.461242 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 15:51:59.464766 22903 catalog_manager.cc:5495] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94640710d80f4186bd98dcc97d749008 (127.15.196.193). New cstate: current_term: 1 leader_uuid: "94640710d80f4186bd98dcc97d749008" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94640710d80f4186bd98dcc97d749008" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 37331 } health_report { overall_health: HEALTHY } } }
I20260929 15:51:59.530799 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.525245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45486 (local address 127.15.196.193:37331)
0929 15:51:59.525453 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:59.525458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:59.525480 (+ 22us) server_negotiation.cc:407] Connection header received
0929 15:51:59.525616 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:59.525620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:59.525694 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:59.525846 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:51:59.526210 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.526916 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.528365 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.528832 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.528931 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.529410 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:51:59.530214 (+ 804us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:51:59.530328 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:59.530411 (+ 83us) server_negotiation.cc:299] Negotiation successful
0929 15:51:59.530479 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 15:51:59.535151 23375 raft_consensus.cc:1261] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f [term 1 FOLLOWER]: Refusing update from remote peer 94640710d80f4186bd98dcc97d749008: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:59.535247 23517 raft_consensus.cc:1261] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7 [term 1 FOLLOWER]: Refusing update from remote peer 94640710d80f4186bd98dcc97d749008: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:51:59.535939 23665 consensus_queue.cc:1035] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cf3ae47c7cb4f09a336e751fa90261f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:59.536325 23660 consensus_queue.cc:1035] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e4767290964f7fa24e766c3c1248b7" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 33907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:51:59.538529 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.538088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.194:45263)
0929 15:51:59.538244 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.538361 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 15:51:59.539806 23661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.539336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.195:52669)
0929 15:51:59.539516 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.539635 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260929 15:51:59.550439 23671 mvcc.cc:205] Tried to move back new op lower bound from 7334695401608159232 to 7334695401317715968. Current Snapshot: MvccSnapshot[applied={T|T < 7334695401608159232}]
I20260929 15:51:59.551715 23661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.550992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.195:36651)
0929 15:51:59.551348 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.551496 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177}
I20260929 15:51:59.553267 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.552507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.193:52789)
0929 15:51:59.552991 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.553108 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381}
I20260929 15:51:59.554493 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.554114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.194:52537)
0929 15:51:59.554277 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.554359 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 15:51:59.556842 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.556290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.15.196.193:49905)
0929 15:51:59.556533 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.556623 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 15:51:59.570080 22904 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:59.570475 22904 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:51:59.579020 22904 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f752fb87cd64afc99afd30cd9da9cee: Sending DeleteTablet for 3 replicas of tablet b77c1d24ce9a46388cd1baece508bef5
I20260929 15:51:59.580092 23208 tablet_service.cc:1514] Processing DeleteTablet for tablet b77c1d24ce9a46388cd1baece508bef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:59 UTC) from {username='oryx', principal='oryx/127.15.196.253@KRBTEST.COM'} at 127.0.0.1:43756
I20260929 15:51:59.580124 23497 tablet_service.cc:1514] Processing DeleteTablet for tablet b77c1d24ce9a46388cd1baece508bef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:59 UTC) from {username='oryx', principal='oryx/127.15.196.253@KRBTEST.COM'} at 127.0.0.1:51436
I20260929 15:51:59.580824 23678 tablet_replica.cc:330] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7: stopping tablet replica
I20260929 15:51:59.580852 23356 tablet_service.cc:1514] Processing DeleteTablet for tablet b77c1d24ce9a46388cd1baece508bef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:59 UTC) from {username='oryx', principal='oryx/127.15.196.253@KRBTEST.COM'} at 127.0.0.1:36310
I20260929 15:51:59.581073 23678 raft_consensus.cc:2229] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:59.581022 23355 tablet_service.cc:1514] Processing DeleteTablet for tablet b77c1d24ce9a46388cd1baece508bef5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:51:59 UTC) from {username='oryx', principal='oryx/127.15.196.253@KRBTEST.COM'} at 127.0.0.1:36310
I20260929 15:51:59.581400 23678 raft_consensus.cc:2258] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:59.582221 23680 tablet_replica.cc:330] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f: stopping tablet replica
I20260929 15:51:59.582417 23680 raft_consensus.cc:2229] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:51:59.582931 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 23080
I20260929 15:51:59.583092 23678 ts_tablet_manager.cc:1905] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:59.583346 23680 raft_consensus.cc:2258] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:59.584388 23680 ts_tablet_manager.cc:1905] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:59.588162 23678 ts_tablet_manager.cc:1918] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:51:59.588268 23678 log.cc:1196] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/b77c1d24ce9a46388cd1baece508bef5
I20260929 15:51:59.588755 23679 tablet_replica.cc:330] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008: stopping tablet replica
I20260929 15:51:59.588806 23678 ts_tablet_manager.cc:1939] T b77c1d24ce9a46388cd1baece508bef5 P d2e4767290964f7fa24e766c3c1248b7: Deleting consensus metadata
I20260929 15:51:59.589068 23679 raft_consensus.cc:2229] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:51:59.589751 23679 raft_consensus.cc:2258] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:51:59.589977 22888 catalog_manager.cc:4841] TS d2e4767290964f7fa24e766c3c1248b7 (127.15.196.195:33907): tablet b77c1d24ce9a46388cd1baece508bef5 (table test-table [id=67cd89da02524b65a235bcec1a0d2321]) successfully deleted
I20260929 15:51:59.591125 23680 ts_tablet_manager.cc:1918] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:51:59.591357 23680 log.cc:1196] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/b77c1d24ce9a46388cd1baece508bef5
I20260929 15:51:59.591385 23679 ts_tablet_manager.cc:1905] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:51:59.591886 23680 ts_tablet_manager.cc:1939] T b77c1d24ce9a46388cd1baece508bef5 P 5cf3ae47c7cb4f09a336e751fa90261f: Deleting consensus metadata
I20260929 15:51:59.592969 22888 catalog_manager.cc:4841] TS 5cf3ae47c7cb4f09a336e751fa90261f (127.15.196.194:33961): tablet b77c1d24ce9a46388cd1baece508bef5 (table test-table [id=67cd89da02524b65a235bcec1a0d2321]) successfully deleted
W20260929 15:51:59.593957 22888 catalog_manager.cc:4808] TS 5cf3ae47c7cb4f09a336e751fa90261f (127.15.196.194:33961): delete failed for tablet b77c1d24ce9a46388cd1baece508bef5 because the tablet was not found. No further retry: Not found: Tablet not found: b77c1d24ce9a46388cd1baece508bef5
I20260929 15:51:59.598543 23679 ts_tablet_manager.cc:1918] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:51:59.598701 23679 log.cc:1196] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/b77c1d24ce9a46388cd1baece508bef5
I20260929 15:51:59.599251 23679 ts_tablet_manager.cc:1939] T b77c1d24ce9a46388cd1baece508bef5 P 94640710d80f4186bd98dcc97d749008: Deleting consensus metadata
I20260929 15:51:59.600442 22891 catalog_manager.cc:4841] TS 94640710d80f4186bd98dcc97d749008 (127.15.196.193:37331): tablet b77c1d24ce9a46388cd1baece508bef5 (table test-table [id=67cd89da02524b65a235bcec1a0d2321]) successfully deleted
I20260929 15:51:59.603600 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.588434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45492 (local address 127.15.196.193:37331)
0929 15:51:59.588653 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:59.588661 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:59.588686 (+ 25us) server_negotiation.cc:407] Connection header received
0929 15:51:59.588757 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:59.588765 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:59.588858 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:59.588976 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:59.590943 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.592183 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.593904 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.594484 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.594564 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.597263 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:59.597280 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:59.597305 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:59.597336 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:59.600856 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:59.601492 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:59.601502 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:59.601506 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:59.601666 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:59.602049 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:59.602054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:59.602057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:59.602382 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:59.602714 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:59.603002 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 15:51:59.603228 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 15:51:59.604229 23268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:59.625221 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.624797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.0.0.1:44724)
0929 15:51:59.624985 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.625084 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260929 15:51:59.626268 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.625943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44195 (local address 127.0.0.1:44734)
0929 15:51:59.626081 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 15:51:59.626151 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 15:51:59.707232 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 23080
I20260929 15:51:59.726549 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 23285
I20260929 15:51:59.740741 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.727465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38568 (local address 127.15.196.194:33961)
0929 15:51:59.727660 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:59.727665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:59.727934 (+ 269us) server_negotiation.cc:407] Connection header received
0929 15:51:59.728103 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:59.728107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:59.728168 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:59.728257 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:59.729797 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.730767 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.732062 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.732459 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.732599 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.735052 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:59.735078 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:59.735096 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:59.735135 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:59.737915 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:59.738582 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:59.738598 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:59.738600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:59.738724 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:59.739112 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:59.739119 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:59.739122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:59.739585 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:59.739880 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:59.740185 (+ 305us) server_negotiation.cc:299] Negotiation successful
0929 15:51:59.740395 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 15:51:59.741460 23416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:59.836136 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 23285
I20260929 15:51:59.854358 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 23434
I20260929 15:51:59.867609 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:51:59.855231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56416 (local address 127.15.196.195:33907)
0929 15:51:59.855420 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 15:51:59.855424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:51:59.855444 (+ 20us) server_negotiation.cc:407] Connection header received
0929 15:51:59.855628 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:51:59.855632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:51:59.855704 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:51:59.855805 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:51:59.857287 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.858022 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.859211 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:51:59.859712 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:51:59.859810 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:51:59.862395 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:51:59.862409 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:51:59.862426 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:51:59.862454 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:51:59.865160 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:59.865768 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:59.865774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:59.865777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:59.865894 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:51:59.866227 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:51:59.866231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:51:59.866233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:51:59.866602 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:51:59.866818 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0929 15:51:59.867057 (+ 239us) server_negotiation.cc:299] Negotiation successful
0929 15:51:59.867221 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260929 15:51:59.868271 23557 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:51:59.989924 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 23434
I20260929 15:52:00.011965 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 22767
I20260929 15:52:00.013128 22934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:00.137631 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 22767
W20260929 15:52:00.160127 23077 connection.cc:535] server connection from 127.0.0.1:41858 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 15:52:00.160548 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 22867
I20260929 15:52:00.161710 23124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:00.279735 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 22867
2026-09-29T15:52:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9767 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:52:00 dist-test-slave-c51v krb5kdc[23689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:52:00 dist-test-slave-c51v krb5kdc[23689](info): set up 2 sockets
Sep 29 15:52:00 dist-test-slave-c51v krb5kdc[23689](info): commencing operation
krb5kdc: starting...
W20260929 15:52:01.398216 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:52:01 dist-test-slave-c51v krb5kdc[23689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:52:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:52:01Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.254.keytab.
Entry for principal oryx/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.254.keytab.
I20260929 15:52:01.615015 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:41821
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.254.keytab
--principal=oryx/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:45635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:41821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:52:01.956956 23702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:01.957293 23702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:01.957448 23702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:01.965145 23702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:52:01.965291 23702 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 15:52:01.965399 23702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:01.965520 23702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:01.965616 23702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:52:01.965744 23702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:52:01.976565 23702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:45635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:41821
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.254.keytab
--principal=oryx/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:41821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:01.977969 23702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:01.979395 23702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:01.991791 23710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:01.991751 23707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:01.991894 23708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:01.992779 23702 server_base.cc:1034] running on GCE node
I20260929 15:52:01.993685 23702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:01.995244 23702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:01.996567 23702 hybrid_clock.cc:648] HybridClock initialized: now 1790697121996494 us; error 46 us; skew 500 ppm
I20260929 15:52:01.996951 23702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:52:01 dist-test-slave-c51v krb5kdc[23689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697121, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:52:02.000093 23702 init.cc:377] Logged in from keytab as oryx/127.15.196.254@KRBTEST.COM (short username oryx)
I20260929 15:52:02.001660 23702 webserver.cc:460] Webserver started at http://127.15.196.254:42231/ using document root <none> and password file <none>
I20260929 15:52:02.002406 23702 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:02.002519 23702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:02.002863 23702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:02.006068 23702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "d5aa785f705747b7abb64b27a8f627c3"
format_stamp: "Formatted at 2026-09-29 15:52:01 on dist-test-slave-c51v"
server_key: "7acbae7b08df194cb067ee1832ab9083"
server_key_iv: "e8ff4ba86e537de3e6964edc68d07c52"
server_key_version: "encryptionkey@0"
I20260929 15:52:02.006913 23702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "d5aa785f705747b7abb64b27a8f627c3"
format_stamp: "Formatted at 2026-09-29 15:52:01 on dist-test-slave-c51v"
server_key: "7acbae7b08df194cb067ee1832ab9083"
server_key_iv: "e8ff4ba86e537de3e6964edc68d07c52"
server_key_version: "encryptionkey@0"
I20260929 15:52:02.012144 23702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 15:52:02.015663 23717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:02.017046 23702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:52:02.017236 23702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "d5aa785f705747b7abb64b27a8f627c3"
format_stamp: "Formatted at 2026-09-29 15:52:01 on dist-test-slave-c51v"
server_key: "7acbae7b08df194cb067ee1832ab9083"
server_key_iv: "e8ff4ba86e537de3e6964edc68d07c52"
server_key_version: "encryptionkey@0"
I20260929 15:52:02.017464 23702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:02.040647 23702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:02.044762 23702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:02.045042 23702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:02.066985 23702 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:41821
I20260929 15:52:02.066989 23778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:41821 every 8 connection(s)
I20260929 15:52:02.068416 23702 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:52:02.072310 23779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:02.076377 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 23702
I20260929 15:52:02.076596 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:52:02.077400 16147 external_mini_cluster.cc:1428] Setting key 50e1845122f533669a4dc4321881baa9
I20260929 15:52:02.084364 23779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3: Bootstrap starting.
I20260929 15:52:02.088169 23779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:02.089319 23779 log.cc:824] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3: Log is configured to *not* fsync() on all Append() calls
I20260929 15:52:02.092866 23779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3: No bootstrap required, opened a new log
Sep 29 15:52:02 dist-test-slave-c51v krb5kdc[23689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM
I20260929 15:52:02.103250 23782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:02.079748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48238 (local address 127.15.196.254:41821)
0929 15:52:02.080692 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:02.080708 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:02.080760 (+ 52us) server_negotiation.cc:407] Connection header received
0929 15:52:02.082887 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:02.082913 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:02.083432 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:02.083806 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:02.085650 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:02.086989 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:02.088756 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:02.089441 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:02.089642 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:02.094320 (+ 4678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:02.094331 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:02.094357 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:02.094406 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:02.098587 (+ 4181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:02.099280 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:02.099318 (+ 38us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:02.099327 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:02.099548 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:02.099912 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:02.099917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:02.099921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:02.100670 (+ 749us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:02.100968 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:02.101297 (+ 329us) server_negotiation.cc:299] Negotiation successful
0929 15:52:02.101784 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":429,"threads_started":1}
I20260929 15:52:02.103881 23779 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aa785f705747b7abb64b27a8f627c3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 41821 } }
I20260929 15:52:02.104254 23779 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:02.104357 23779 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5aa785f705747b7abb64b27a8f627c3, State: Initialized, Role: FOLLOWER
I20260929 15:52:02.104907 23779 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aa785f705747b7abb64b27a8f627c3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 41821 } }
I20260929 15:52:02.105180 23779 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:52:02.105336 23779 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:52:02.105522 23779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:02.107846 23779 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aa785f705747b7abb64b27a8f627c3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 41821 } }
I20260929 15:52:02.108367 23779 leader_election.cc:302] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5aa785f705747b7abb64b27a8f627c3; no voters:
I20260929 15:52:02.108911 23779 leader_election.cc:288] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:52:02.109048 23784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:52:02.109800 23784 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [term 1 LEADER]: Becoming Leader. State: Replica: d5aa785f705747b7abb64b27a8f627c3, State: Running, Role: LEADER
I20260929 15:52:02.110330 23784 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aa785f705747b7abb64b27a8f627c3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 41821 } }
I20260929 15:52:02.110961 23779 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:52:02.113511 23786 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5aa785f705747b7abb64b27a8f627c3. Latest consensus state: current_term: 1 leader_uuid: "d5aa785f705747b7abb64b27a8f627c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aa785f705747b7abb64b27a8f627c3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 41821 } } }
I20260929 15:52:02.113490 23785 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5aa785f705747b7abb64b27a8f627c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aa785f705747b7abb64b27a8f627c3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 41821 } } }
I20260929 15:52:02.114070 23785 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [sys.catalog]: This master's current role is: LEADER
I20260929 15:52:02.114918 23786 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3 [sys.catalog]: This master's current role is: LEADER
I20260929 15:52:02.121431 23791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:52:02.126734 23791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:52:02.135983 23791 catalog_manager.cc:1347] Generated new cluster ID: ad08a60516a2449291e5c2401530c096
I20260929 15:52:02.136114 23791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:52:02.152262 23791 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:52:02.153694 23791 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:52:02.160392 23791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3: Generated new TSK 0
I20260929 15:52:02.161221 23791 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.193.keytab.
Entry for principal oryx/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.193.keytab.
I20260929 15:52:02.222671 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:41821
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.193.keytab
--principal=oryx/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:45635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:52:02.574910 23805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:02.575215 23805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:02.575418 23805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:02.582823 23805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:02.582942 23805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:02.583088 23805 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 15:52:02.583216 23805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:52:02.593257 23805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:45635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.193.keytab
--principal=oryx/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.196.254:41821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:02.594620 23805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:02.596095 23805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:02.609625 23810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:02.609534 23813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:02.609592 23811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:02.611205 23805 server_base.cc:1034] running on GCE node
I20260929 15:52:02.611961 23805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:02.613085 23805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:02.614331 23805 hybrid_clock.cc:648] HybridClock initialized: now 1790697122614235 us; error 63 us; skew 500 ppm
I20260929 15:52:02.614621 23805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:52:02 dist-test-slave-c51v krb5kdc[23689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697122, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:52:02.618057 23805 init.cc:377] Logged in from keytab as oryx/127.15.196.193@KRBTEST.COM (short username oryx)
I20260929 15:52:02.619709 23805 webserver.cc:460] Webserver started at http://127.15.196.193:39169/ using document root <none> and password file <none>
I20260929 15:52:02.620443 23805 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:02.620559 23805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:02.620958 23805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:02.624091 23805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "d72220c68c774184b366c2dacdca8f76"
format_stamp: "Formatted at 2026-09-29 15:52:02 on dist-test-slave-c51v"
server_key: "420b2b5391cb46ff287be43ef6bea5ea"
server_key_iv: "157af395b74d6615d102dcffde0b872f"
server_key_version: "encryptionkey@0"
I20260929 15:52:02.624902 23805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "d72220c68c774184b366c2dacdca8f76"
format_stamp: "Formatted at 2026-09-29 15:52:02 on dist-test-slave-c51v"
server_key: "420b2b5391cb46ff287be43ef6bea5ea"
server_key_iv: "157af395b74d6615d102dcffde0b872f"
server_key_version: "encryptionkey@0"
I20260929 15:52:02.630087 23805 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:52:02.633533 23820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:02.634866 23805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:52:02.635046 23805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "d72220c68c774184b366c2dacdca8f76"
format_stamp: "Formatted at 2026-09-29 15:52:02 on dist-test-slave-c51v"
server_key: "420b2b5391cb46ff287be43ef6bea5ea"
server_key_iv: "157af395b74d6615d102dcffde0b872f"
server_key_version: "encryptionkey@0"
I20260929 15:52:02.635260 23805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:02.657774 23805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:02.662113 23805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:02.662408 23805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:02.664995 23805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:02.665082 23805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:02.665246 23805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:02.665301 23805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 15:52:02 dist-test-slave-c51v krb5kdc[23689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697122, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM
I20260929 15:52:02.688155 23829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:02.665742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:43103 (local address 127.15.196.254:41821)
0929 15:52:02.666251 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:02.666257 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:02.667430 (+ 1173us) server_negotiation.cc:407] Connection header received
0929 15:52:02.669455 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:02.669461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:02.669560 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:02.669675 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:02.671876 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:02.672944 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:02.674559 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:02.675142 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:02.675310 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:02.680201 (+ 4891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:02.680207 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:02.680210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:02.680242 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:02.683753 (+ 3511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:02.684797 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:02.684803 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:02.684807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:02.684966 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:02.685506 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:02.685511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:02.685514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:02.685909 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:02.686088 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:02.687232 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0929 15:52:02.687597 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":263,"threads_started":1}
I20260929 15:52:02.688809 23830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:02.666350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:41821 (local address 127.15.196.193:43103)
0929 15:52:02.667238 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:02.667310 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:02.669165 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:02.669916 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:02.669924 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:02.670679 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:02.671622 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:02.671655 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:02.673127 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:02.673131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:02.674412 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:02.674423 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:02.675493 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:02.675505 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:02.675871 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:02.676795 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:02.676835 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:02.679929 (+ 3094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:02.683912 (+ 3983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:02.683924 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:02.683951 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:02.684642 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:02.685098 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:02.685105 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:02.685110 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:02.685377 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:02.686062 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:02.686070 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:02.686666 (+ 596us) client_negotiation.cc:769] Sending connection context
0929 15:52:02.687047 (+ 381us) client_negotiation.cc:240] Negotiation successful
0929 15:52:02.687581 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":344,"threads_started":1}
I20260929 15:52:02.711596 23829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:02.695511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:56707 (local address 127.15.196.254:41821)
0929 15:52:02.695747 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:02.695756 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:02.696321 (+ 565us) server_negotiation.cc:407] Connection header received
0929 15:52:02.696563 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:02.696569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:02.696638 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:02.696693 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:02.698503 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:02.699944 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:02.702070 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:02.702767 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:02.702920 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:02.705472 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:02.705481 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:02.705489 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:02.705547 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:02.708969 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:02.709610 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:02.709616 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:02.709620 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:02.709747 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:02.710104 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:02.710119 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:02.710124 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:02.710385 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:02.710566 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:02.710938 (+ 372us) server_negotiation.cc:299] Negotiation successful
0929 15:52:02.711191 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260929 15:52:02.711719 23898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:02.695591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:41821 (local address 127.15.196.193:56707)
0929 15:52:02.696205 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:02.696230 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:02.696387 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:02.696908 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:02.696914 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:02.697854 (+ 940us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:02.698328 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:02.698347 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:02.700115 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:02.700122 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:02.701949 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:02.701962 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:02.703005 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:02.703012 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:02.703248 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:02.704004 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:02.704032 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:02.705173 (+ 1141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:02.709101 (+ 3928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:02.709106 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:02.709111 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:02.709488 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:02.709854 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:02.709860 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:02.709864 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:02.709991 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:02.710501 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:02.710506 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:02.710698 (+ 192us) client_negotiation.cc:769] Sending connection context
0929 15:52:02.710837 (+ 139us) client_negotiation.cc:240] Negotiation successful
0929 15:52:02.711183 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":327,"threads_started":1}
I20260929 15:52:02.713788 23805 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:33007
I20260929 15:52:02.713812 23940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:33007 every 8 connection(s)
I20260929 15:52:02.715312 23805 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:52:02.717401 23941 heartbeater.cc:347] Connected to a master server at 127.15.196.254:41821
I20260929 15:52:02.717656 23941 heartbeater.cc:464] Registering TS with master...
I20260929 15:52:02.718284 23941 heartbeater.cc:511] Master 127.15.196.254:41821 requested a full tablet report, sending...
I20260929 15:52:02.720082 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 23805
I20260929 15:52:02.720288 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:52:02.720237 23734 ts_manager.cc:194] Registered new tserver with Master: d72220c68c774184b366c2dacdca8f76 (127.15.196.193:33007)
I20260929 15:52:02.720922 16147 external_mini_cluster.cc:1428] Setting key 68210179bbe16cd50251ce14dc948fc0
I20260929 15:52:02.722129 23734 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:43103
WARNING: no policy specified for oryx/127.15.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.194.keytab.
Entry for principal oryx/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.194.keytab.
I20260929 15:52:02.770364 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:41821
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.194.keytab
--principal=oryx/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:45635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:52:03.105746 23945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:03.106055 23945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:03.106158 23945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:03.113770 23945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:03.113914 23945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:03.113999 23945 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 15:52:03.114079 23945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:52:03.124737 23945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:45635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.194.keytab
--principal=oryx/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.196.254:41821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:03.126112 23945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:03.127516 23945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:03.140826 23953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:03.141348 23950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:03.141433 23951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:03.141834 23945 server_base.cc:1034] running on GCE node
I20260929 15:52:03.142447 23945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:03.143525 23945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:03.144783 23945 hybrid_clock.cc:648] HybridClock initialized: now 1790697123144712 us; error 54 us; skew 500 ppm
I20260929 15:52:03.145087 23945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:52:03 dist-test-slave-c51v krb5kdc[23689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697123, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:52:03.148311 23945 init.cc:377] Logged in from keytab as oryx/127.15.196.194@KRBTEST.COM (short username oryx)
I20260929 15:52:03.149731 23945 webserver.cc:460] Webserver started at http://127.15.196.194:45395/ using document root <none> and password file <none>
I20260929 15:52:03.150454 23945 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:03.150592 23945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:03.150942 23945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:03.153998 23945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "af29a2e9090c4a5fb36fe839cdd4646b"
format_stamp: "Formatted at 2026-09-29 15:52:03 on dist-test-slave-c51v"
server_key: "7632e024db0bc2f5dedd86b4a9c27063"
server_key_iv: "4f8753f43520f447584e421f34f0d290"
server_key_version: "encryptionkey@0"
I20260929 15:52:03.155107 23945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "af29a2e9090c4a5fb36fe839cdd4646b"
format_stamp: "Formatted at 2026-09-29 15:52:03 on dist-test-slave-c51v"
server_key: "7632e024db0bc2f5dedd86b4a9c27063"
server_key_iv: "4f8753f43520f447584e421f34f0d290"
server_key_version: "encryptionkey@0"
I20260929 15:52:03.160805 23945 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 15:52:03.164505 23960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:03.165910 23945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:52:03.166112 23945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "af29a2e9090c4a5fb36fe839cdd4646b"
format_stamp: "Formatted at 2026-09-29 15:52:03 on dist-test-slave-c51v"
server_key: "7632e024db0bc2f5dedd86b4a9c27063"
server_key_iv: "4f8753f43520f447584e421f34f0d290"
server_key_version: "encryptionkey@0"
I20260929 15:52:03.166373 23945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:03.186525 23945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:03.191457 23945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:03.191746 23945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:03.194612 23945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:03.194720 23945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:03.194844 23945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:03.194923 23945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 15:52:03 dist-test-slave-c51v krb5kdc[23689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697123, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM
I20260929 15:52:03.221962 23829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.195199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:54363 (local address 127.15.196.254:41821)
0929 15:52:03.195480 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:03.195488 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:03.197121 (+ 1633us) server_negotiation.cc:407] Connection header received
0929 15:52:03.199431 (+ 2310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:03.199437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:03.199536 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:03.199685 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:03.202828 (+ 3143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.203975 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.207417 (+ 3442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.208094 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.208255 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:03.213638 (+ 5383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:03.213645 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:03.213652 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:03.213694 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:03.217244 (+ 3550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:03.218449 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:03.218455 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:03.218457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:03.218609 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:03.219163 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:03.219169 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:03.219180 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:03.219573 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:03.219762 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:03.221108 (+ 1346us) server_negotiation.cc:299] Negotiation successful
0929 15:52:03.221458 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260929 15:52:03.223157 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.195984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:41821 (local address 127.15.196.194:54363)
0929 15:52:03.196891 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:03.196937 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:03.199059 (+ 2122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:03.200170 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:03.200186 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:03.201391 (+ 1205us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:03.202541 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.202582 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.204166 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.204179 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:03.207203 (+ 3024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.207225 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.208378 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.208390 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:03.208945 (+ 555us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:03.209924 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:03.209965 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:03.213376 (+ 3411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:03.217429 (+ 4053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:03.217438 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:03.217475 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:03.218003 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:03.218749 (+ 746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:03.218755 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:03.218759 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:03.219016 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:03.219720 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:03.219731 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:03.220435 (+ 704us) client_negotiation.cc:769] Sending connection context
0929 15:52:03.220904 (+ 469us) client_negotiation.cc:240] Negotiation successful
0929 15:52:03.221530 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":386,"threads_started":1}
I20260929 15:52:03.246131 23829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.229616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:41273 (local address 127.15.196.254:41821)
0929 15:52:03.229894 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:03.229903 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:03.230719 (+ 816us) server_negotiation.cc:407] Connection header received
0929 15:52:03.230954 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:03.230967 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:03.231033 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:03.231144 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:03.232177 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.233005 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.234705 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.235514 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.235638 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:03.239034 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:03.239042 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:03.239048 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:03.239101 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:03.243064 (+ 3963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:03.243900 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:03.243909 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:03.243914 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:03.244062 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:03.244491 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:03.244500 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:03.244507 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:03.244872 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:03.245053 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:03.245485 (+ 432us) server_negotiation.cc:299] Negotiation successful
0929 15:52:03.245742 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260929 15:52:03.246270 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.229617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:41821 (local address 127.15.196.194:41273)
0929 15:52:03.230584 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:03.230614 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:03.230801 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:03.231168 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:03.231173 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:03.231718 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:03.232024 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.232034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.233162 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.233168 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:03.234487 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.234506 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.235964 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.235972 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:03.236240 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:03.237192 (+ 952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:03.237217 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:03.238643 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:03.243228 (+ 4585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:03.243234 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:03.243239 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:03.243763 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:03.244190 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:03.244196 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:03.244200 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:03.244361 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:03.245010 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:03.245016 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:03.245220 (+ 204us) client_negotiation.cc:769] Sending connection context
0929 15:52:03.245386 (+ 166us) client_negotiation.cc:240] Negotiation successful
0929 15:52:03.245759 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"thread_start_us":634,"threads_started":1}
I20260929 15:52:03.256007 23945 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:38361
I20260929 15:52:03.256036 24079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:38361 every 8 connection(s)
I20260929 15:52:03.257385 23945 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:52:03.259539 24080 heartbeater.cc:347] Connected to a master server at 127.15.196.254:41821
I20260929 15:52:03.259744 24080 heartbeater.cc:464] Registering TS with master...
I20260929 15:52:03.260380 24080 heartbeater.cc:511] Master 127.15.196.254:41821 requested a full tablet report, sending...
I20260929 15:52:03.261767 23734 ts_manager.cc:194] Registered new tserver with Master: af29a2e9090c4a5fb36fe839cdd4646b (127.15.196.194:38361)
I20260929 15:52:03.262235 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 23945
I20260929 15:52:03.262411 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:52:03.262941 16147 external_mini_cluster.cc:1428] Setting key 5c18ca0ef121e8dff4f7ac9e83e85a49
I20260929 15:52:03.262990 23734 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:54363
WARNING: no policy specified for oryx/127.15.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.195.keytab.
Entry for principal oryx/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.195.keytab.
I20260929 15:52:03.311097 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:41821
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.195.keytab
--principal=oryx/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:45635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:52:03.645596 24084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:03.645939 24084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:03.646116 24084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:03.653575 24084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:03.653748 24084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:03.653899 24084 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 15:52:03.654038 24084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:52:03.664472 24084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:45635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.195.keytab
--principal=oryx/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.196.254:41821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:03.665963 24084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:03.667400 24084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:03.680878 24090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:03.680966 24092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:03.681180 24089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:03.682906 24084 server_base.cc:1034] running on GCE node
I20260929 15:52:03.683657 24084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:03.684679 24084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:03.685933 24084 hybrid_clock.cc:648] HybridClock initialized: now 1790697123685875 us; error 47 us; skew 500 ppm
I20260929 15:52:03.686275 24084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:52:03 dist-test-slave-c51v krb5kdc[23689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697123, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:52:03.689590 24084 init.cc:377] Logged in from keytab as oryx/127.15.196.195@KRBTEST.COM (short username oryx)
I20260929 15:52:03.691326 24084 webserver.cc:460] Webserver started at http://127.15.196.195:37261/ using document root <none> and password file <none>
I20260929 15:52:03.692117 24084 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:03.692247 24084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:03.692596 24084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:03.695639 24084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "563a2b2e105b4346b04102d7f35be39c"
format_stamp: "Formatted at 2026-09-29 15:52:03 on dist-test-slave-c51v"
server_key: "f58126873d61e708ab81065796086bad"
server_key_iv: "5fd2ae039481688a8d08ec6ab63238eb"
server_key_version: "encryptionkey@0"
I20260929 15:52:03.696444 24084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "563a2b2e105b4346b04102d7f35be39c"
format_stamp: "Formatted at 2026-09-29 15:52:03 on dist-test-slave-c51v"
server_key: "f58126873d61e708ab81065796086bad"
server_key_iv: "5fd2ae039481688a8d08ec6ab63238eb"
server_key_version: "encryptionkey@0"
I20260929 15:52:03.701819 24084 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:52:03.705405 24099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:03.706720 24084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:52:03.706902 24084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "563a2b2e105b4346b04102d7f35be39c"
format_stamp: "Formatted at 2026-09-29 15:52:03 on dist-test-slave-c51v"
server_key: "f58126873d61e708ab81065796086bad"
server_key_iv: "5fd2ae039481688a8d08ec6ab63238eb"
server_key_version: "encryptionkey@0"
I20260929 15:52:03.707125 24084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:03.725914 23941 heartbeater.cc:503] Master 127.15.196.254:41821 was elected leader, sending a full tablet report...
I20260929 15:52:03.740341 24084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:03.744268 24084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:03.744553 24084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:03.747530 24084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:03.747651 24084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:03.747819 24084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:03.747905 24084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 15:52:03 dist-test-slave-c51v krb5kdc[23689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697123, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM
I20260929 15:52:03.769618 24107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.747842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:48973 (local address 127.15.196.254:41821)
0929 15:52:03.748385 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:03.748391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:03.749599 (+ 1208us) server_negotiation.cc:407] Connection header received
0929 15:52:03.751867 (+ 2268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:03.751872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:03.751968 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:03.752095 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:03.754495 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.755559 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.758041 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.758501 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.758648 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:03.763373 (+ 4725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:03.763378 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:03.763381 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:03.763420 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:03.766233 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:03.766983 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:03.766991 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:03.766999 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:03.767132 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:03.767615 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:03.767623 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:03.767629 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:03.768056 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:03.768211 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:03.768998 (+ 787us) server_negotiation.cc:299] Negotiation successful
0929 15:52:03.769227 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":261,"threads_started":1}
I20260929 15:52:03.770766 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.748580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:41821 (local address 127.15.196.195:48973)
0929 15:52:03.749415 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:03.749470 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:03.751585 (+ 2115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:03.752584 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:03.752593 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:03.753418 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:03.754322 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.754341 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.755720 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.755735 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:03.757878 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.757890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.758948 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.758959 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:03.759264 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:03.760116 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:03.760159 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:03.763098 (+ 2939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:03.766392 (+ 3294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:03.766401 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:03.766419 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:03.766850 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:03.767247 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:03.767253 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:03.767257 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:03.767484 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:03.768199 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:03.768207 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:03.768617 (+ 410us) client_negotiation.cc:769] Sending connection context
0929 15:52:03.768879 (+ 262us) client_negotiation.cc:240] Negotiation successful
0929 15:52:03.769451 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":377,"threads_started":1}
I20260929 15:52:03.798386 24107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.782579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:45305 (local address 127.15.196.254:41821)
0929 15:52:03.782785 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:03.782793 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:03.783692 (+ 899us) server_negotiation.cc:407] Connection header received
0929 15:52:03.783961 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:03.783969 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:03.784080 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:03.784215 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:03.785655 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.786691 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.788233 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.788801 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.788954 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:03.791638 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:03.791645 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:03.791653 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:03.791697 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:03.795496 (+ 3799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:03.796404 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:03.796409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:03.796413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:03.796576 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:03.797061 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:03.797067 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:03.797073 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:03.797363 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:03.797479 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:03.797939 (+ 460us) server_negotiation.cc:299] Negotiation successful
0929 15:52:03.798136 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 15:52:03.798837 24163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.782675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:41821 (local address 127.15.196.195:45305)
0929 15:52:03.783559 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:03.783582 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:03.783769 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:03.784246 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:03.784251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:03.784930 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:03.785499 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.785519 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.786832 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.786841 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:03.788007 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.788021 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.789030 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.789046 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:03.789241 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:03.789921 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:03.789953 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:03.791353 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:03.795675 (+ 4322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:03.795679 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:03.795681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:03.796269 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:03.796694 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:03.796704 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:03.796710 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:03.796915 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:03.797455 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:03.797460 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:03.797677 (+ 217us) client_negotiation.cc:769] Sending connection context
0929 15:52:03.797842 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 15:52:03.798236 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":468,"threads_started":1}
I20260929 15:52:03.809157 24084 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:42727
I20260929 15:52:03.809186 24219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:42727 every 8 connection(s)
I20260929 15:52:03.810410 24084 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:52:03.812554 24220 heartbeater.cc:347] Connected to a master server at 127.15.196.254:41821
I20260929 15:52:03.812788 24220 heartbeater.cc:464] Registering TS with master...
I20260929 15:52:03.813341 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 24084
I20260929 15:52:03.813520 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:52:03.813598 24220 heartbeater.cc:511] Master 127.15.196.254:41821 requested a full tablet report, sending...
I20260929 15:52:03.814097 16147 external_mini_cluster.cc:1428] Setting key dfab0cad174bcd2281ab2c7dbc224187
I20260929 15:52:03.815065 23734 ts_manager.cc:194] Registered new tserver with Master: 563a2b2e105b4346b04102d7f35be39c (127.15.196.195:42727)
I20260929 15:52:03.816067 23734 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:48973
I20260929 15:52:03.817086 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 15:52:03 dist-test-slave-c51v krb5kdc[23689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697123, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 15:52:03 dist-test-slave-c51v krb5kdc[23689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:03 dist-test-slave-c51v krb5kdc[23689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 15:52:03.865483 24107 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:03.855421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48250 (local address 127.15.196.254:41821)
0929 15:52:03.855651 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:03.855657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:03.855819 (+ 162us) server_negotiation.cc:407] Connection header received
0929 15:52:03.855984 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:03.855988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:03.856054 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:03.856139 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:03.857385 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.858155 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.859504 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.859890 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.859983 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:03.865199 (+ 5216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48250: BlockingRecv error: recv got EOF from 127.0.0.1:48250 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 29 15:52:03 dist-test-slave-c51v krb5kdc[23689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697123, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM
I20260929 15:52:03.883147 24107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.868673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48264 (local address 127.15.196.254:41821)
0929 15:52:03.868896 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:03.868903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:03.869157 (+ 254us) server_negotiation.cc:407] Connection header received
0929 15:52:03.869372 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:03.869376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:03.869434 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:03.869536 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:03.870670 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.871408 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.872914 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.873445 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.873600 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:03.877613 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:03.877619 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:03.877623 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:03.877659 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:03.880335 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:03.881049 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:03.881056 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:03.881060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:03.881200 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:03.881620 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:03.881626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:03.881630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:03.881918 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:03.882188 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:03.882571 (+ 383us) server_negotiation.cc:299] Negotiation successful
0929 15:52:03.882821 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260929 15:52:03.887264 23733 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:52:03.894510 23733 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:52:03.927829 24241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.907126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55758 (local address 127.15.196.194:38361)
0929 15:52:03.919103 (+ 11977us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:03.919117 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:03.919160 (+ 43us) server_negotiation.cc:407] Connection header received
0929 15:52:03.919311 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:03.919324 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:03.919700 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:03.920808 (+ 1108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:52:03.921021 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.922485 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.925181 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.926668 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.926875 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:03.927074 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:03.927341 (+ 267us) server_negotiation.cc:299] Negotiation successful
0929 15:52:03.927514 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11738,"thread_start_us":12761,"threads_started":1}
I20260929 15:52:03.927901 24240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.906892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:38361 (local address 127.0.0.1:55758)
0929 15:52:03.918573 (+ 11681us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:03.918597 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:03.918746 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:03.920248 (+ 1502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:03.920261 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:03.920316 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:52:03.920823 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.920842 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.922611 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.922619 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:03.924996 (+ 2377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.925007 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.926875 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.926881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:03.927059 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:03.927125 (+ 66us) client_negotiation.cc:769] Sending connection context
0929 15:52:03.927289 (+ 164us) client_negotiation.cc:240] Negotiation successful
0929 15:52:03.927522 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11481,"mutex_wait_us":2,"thread_start_us":11962,"threads_started":1}
I20260929 15:52:03.929085 24238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.906199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:42727 (local address 127.0.0.1:56452)
0929 15:52:03.913548 (+ 7349us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:03.913615 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:03.913885 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:03.921500 (+ 7615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:03.921504 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:03.921529 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:52:03.922091 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.922104 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.923799 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.923805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:03.926124 (+ 2319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.926138 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.928274 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.928280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:03.928443 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:03.928482 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 15:52:03.928563 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 15:52:03.928697 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6972,"mutex_wait_us":74,"thread_start_us":474,"threads_started":1}
I20260929 15:52:03.929401 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.906528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56452 (local address 127.15.196.195:42727)
0929 15:52:03.920881 (+ 14353us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:03.920892 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:03.920924 (+ 32us) server_negotiation.cc:407] Connection header received
0929 15:52:03.921005 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:03.921011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:03.921232 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:03.922069 (+ 837us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:52:03.922383 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.923665 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.926307 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.927763 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.927951 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:03.928128 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:03.928878 (+ 750us) server_negotiation.cc:299] Negotiation successful
0929 15:52:03.929057 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14165,"thread_start_us":13959,"threads_started":1}
I20260929 15:52:03.929997 24242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.907564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:33007 (local address 127.0.0.1:37970)
0929 15:52:03.920469 (+ 12905us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:03.920491 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:03.920747 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:03.921450 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:03.921454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:03.921496 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:52:03.922372 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.922380 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.924637 (+ 2257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.924642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:03.926830 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.926843 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.928655 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.928661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:03.928824 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:03.928853 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 15:52:03.928944 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 15:52:03.929091 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12754,"mutex_wait_us":44,"thread_start_us":12592,"threads_started":1}
I20260929 15:52:03.930779 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.908089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37970 (local address 127.15.196.193:33007)
0929 15:52:03.919950 (+ 11861us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:03.919963 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:03.920651 (+ 688us) server_negotiation.cc:407] Connection header received
0929 15:52:03.920989 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:03.920997 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:03.921232 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:03.921427 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:52:03.922934 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.924472 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.927007 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.928434 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.928622 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:03.928769 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:03.929196 (+ 427us) server_negotiation.cc:299] Negotiation successful
0929 15:52:03.930465 (+ 1269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11702,"thread_start_us":11570,"threads_started":1}
I20260929 15:52:03.932312 24010 tablet_service.cc:1467] Processing CreateTablet for tablet 29fa3c20d67f45949b9d1634c2c17d67 (DEFAULT_TABLE table=test-table [id=36850c23cab34fd4be74daedf8129fdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:52:03.932765 24154 tablet_service.cc:1467] Processing CreateTablet for tablet 29fa3c20d67f45949b9d1634c2c17d67 (DEFAULT_TABLE table=test-table [id=36850c23cab34fd4be74daedf8129fdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:52:03.933808 24010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29fa3c20d67f45949b9d1634c2c17d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:03.933991 24154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29fa3c20d67f45949b9d1634c2c17d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:03.934064 23871 tablet_service.cc:1467] Processing CreateTablet for tablet 29fa3c20d67f45949b9d1634c2c17d67 (DEFAULT_TABLE table=test-table [id=36850c23cab34fd4be74daedf8129fdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:52:03.935390 23871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29fa3c20d67f45949b9d1634c2c17d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:03.946204 24244 tablet_bootstrap.cc:492] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c: Bootstrap starting.
I20260929 15:52:03.949285 24244 tablet_bootstrap.cc:654] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:03.950096 24246 tablet_bootstrap.cc:492] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76: Bootstrap starting.
I20260929 15:52:03.950666 24244 log.cc:824] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c: Log is configured to *not* fsync() on all Append() calls
I20260929 15:52:03.953238 24245 tablet_bootstrap.cc:492] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b: Bootstrap starting.
I20260929 15:52:03.953756 24246 tablet_bootstrap.cc:654] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:03.954078 24244 tablet_bootstrap.cc:492] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c: No bootstrap required, opened a new log
I20260929 15:52:03.954373 24244 ts_tablet_manager.cc:1397] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.003s
I20260929 15:52:03.955036 24246 log.cc:824] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76: Log is configured to *not* fsync() on all Append() calls
I20260929 15:52:03.956445 24245 tablet_bootstrap.cc:654] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:03.957653 24245 log.cc:824] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b: Log is configured to *not* fsync() on all Append() calls
I20260929 15:52:03.958118 24246 tablet_bootstrap.cc:492] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76: No bootstrap required, opened a new log
I20260929 15:52:03.958446 24246 ts_tablet_manager.cc:1397] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260929 15:52:03.960619 24245 tablet_bootstrap.cc:492] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b: No bootstrap required, opened a new log
I20260929 15:52:03.960924 24245 ts_tablet_manager.cc:1397] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260929 15:52:03.962790 24244 raft_consensus.cc:348] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } }
I20260929 15:52:03.963161 24244 raft_consensus.cc:374] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:03.963318 24244 raft_consensus.cc:729] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 563a2b2e105b4346b04102d7f35be39c, State: Initialized, Role: FOLLOWER
I20260929 15:52:03.963949 24244 consensus_queue.cc:260] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } }
I20260929 15:52:03.965013 24220 heartbeater.cc:503] Master 127.15.196.254:41821 was elected leader, sending a full tablet report...
I20260929 15:52:03.966962 24244 ts_tablet_manager.cc:1428] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c: Time spent starting tablet: real 0.012s user 0.006s sys 0.005s
I20260929 15:52:03.968957 24246 raft_consensus.cc:348] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } }
I20260929 15:52:03.969391 24246 raft_consensus.cc:374] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:03.969494 24246 raft_consensus.cc:729] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72220c68c774184b366c2dacdca8f76, State: Initialized, Role: FOLLOWER
I20260929 15:52:03.970211 24246 consensus_queue.cc:260] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } }
I20260929 15:52:03.971323 24245 raft_consensus.cc:348] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } }
I20260929 15:52:03.971810 24245 raft_consensus.cc:374] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:03.971946 24245 raft_consensus.cc:729] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af29a2e9090c4a5fb36fe839cdd4646b, State: Initialized, Role: FOLLOWER
I20260929 15:52:03.972020 24246 ts_tablet_manager.cc:1428] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260929 15:52:03.972694 24245 consensus_queue.cc:260] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } }
I20260929 15:52:03.973914 24080 heartbeater.cc:503] Master 127.15.196.254:41821 was elected leader, sending a full tablet report...
I20260929 15:52:03.974400 24245 ts_tablet_manager.cc:1428] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260929 15:52:03.992890 24252 raft_consensus.cc:482] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:52:03.993266 24252 raft_consensus.cc:504] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } }
I20260929 15:52:03.995061 24252 leader_election.cc:288] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 563a2b2e105b4346b04102d7f35be39c (127.15.196.195:42727), d72220c68c774184b366c2dacdca8f76 (127.15.196.193:33007)
I20260929 15:52:04.001588 24253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.995473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:42727 (local address 127.15.196.194:39595)
0929 15:52:03.996152 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:03.996173 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:03.996311 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:03.996726 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:03.996731 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:03.996757 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:52:03.997170 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.997179 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.998171 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.998179 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:03.999663 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.999674 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.000767 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.000773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:04.000945 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.000969 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 15:52:04.001087 (+ 118us) client_negotiation.cc:240] Negotiation successful
0929 15:52:04.001208 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":354,"threads_started":1}
I20260929 15:52:04.002038 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.995418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:39595 (local address 127.15.196.195:42727)
0929 15:52:03.995633 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:03.995638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:03.996275 (+ 637us) server_negotiation.cc:407] Connection header received
0929 15:52:03.996478 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:03.996483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:03.996590 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:03.996702 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:52:03.997313 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.998045 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.999817 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.000561 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.000748 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.000825 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:04.001651 (+ 826us) server_negotiation.cc:299] Negotiation successful
0929 15:52:04.001791 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260929 15:52:04.002243 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.996459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:44633 (local address 127.15.196.193:33007)
0929 15:52:03.996691 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:03.996696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:03.996992 (+ 296us) server_negotiation.cc:407] Connection header received
0929 15:52:03.997197 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:03.997204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:03.997275 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:03.997368 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:52:03.997882 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.998571 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.000276 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.001289 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.001470 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.001553 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:04.001859 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 15:52:04.001982 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 15:52:04.002884 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:03.996280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:33007 (local address 127.15.196.194:44633)
0929 15:52:03.996884 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:03.996898 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:03.997066 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:03.997381 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:03.997385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:03.997405 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:52:03.997729 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:03.997737 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:03.998715 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:03.998720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:04.000124 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:04.000135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.001502 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.001508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:04.001685 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.001716 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 15:52:04.002409 (+ 693us) client_negotiation.cc:240] Negotiation successful
0929 15:52:04.002562 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"spinlock_wait_cycles":20736,"thread_start_us":411,"threads_started":1}
I20260929 15:52:04.003120 24175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fa3c20d67f45949b9d1634c2c17d67" candidate_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "563a2b2e105b4346b04102d7f35be39c" is_pre_election: true
I20260929 15:52:04.003705 24175 raft_consensus.cc:2454] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af29a2e9090c4a5fb36fe839cdd4646b in term 0.
I20260929 15:52:04.003800 23895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fa3c20d67f45949b9d1634c2c17d67" candidate_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72220c68c774184b366c2dacdca8f76" is_pre_election: true
I20260929 15:52:04.004313 23895 raft_consensus.cc:2454] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af29a2e9090c4a5fb36fe839cdd4646b in term 0.
I20260929 15:52:04.004308 23962 leader_election.cc:302] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 563a2b2e105b4346b04102d7f35be39c, af29a2e9090c4a5fb36fe839cdd4646b; no voters:
I20260929 15:52:04.004709 24252 raft_consensus.cc:2781] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:52:04.004861 24252 raft_consensus.cc:482] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:52:04.004998 24252 raft_consensus.cc:3037] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:04.007126 24252 raft_consensus.cc:504] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } }
I20260929 15:52:04.007689 24252 leader_election.cc:288] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [CANDIDATE]: Term 1 election: Requested vote from peers 563a2b2e105b4346b04102d7f35be39c (127.15.196.195:42727), d72220c68c774184b366c2dacdca8f76 (127.15.196.193:33007)
I20260929 15:52:04.008157 24175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fa3c20d67f45949b9d1634c2c17d67" candidate_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "563a2b2e105b4346b04102d7f35be39c"
I20260929 15:52:04.008368 24175 raft_consensus.cc:3037] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:04.008427 23895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29fa3c20d67f45949b9d1634c2c17d67" candidate_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72220c68c774184b366c2dacdca8f76"
I20260929 15:52:04.008636 23895 raft_consensus.cc:3037] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Advancing to term 1
W20260929 15:52:04.009661 24081 tablet.cc:2369] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:52:04.011770 23895 raft_consensus.cc:2454] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af29a2e9090c4a5fb36fe839cdd4646b in term 1.
I20260929 15:52:04.011770 24175 raft_consensus.cc:2454] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af29a2e9090c4a5fb36fe839cdd4646b in term 1.
I20260929 15:52:04.012177 23962 leader_election.cc:302] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af29a2e9090c4a5fb36fe839cdd4646b, d72220c68c774184b366c2dacdca8f76; no voters:
I20260929 15:52:04.012455 24252 raft_consensus.cc:2781] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:52:04.012945 24252 raft_consensus.cc:686] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [term 1 LEADER]: Becoming Leader. State: Replica: af29a2e9090c4a5fb36fe839cdd4646b, State: Running, Role: LEADER
I20260929 15:52:04.013394 24252 consensus_queue.cc:237] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } }
I20260929 15:52:04.017830 23733 catalog_manager.cc:5495] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b reported cstate change: term changed from 0 to 1, leader changed from <none> to af29a2e9090c4a5fb36fe839cdd4646b (127.15.196.194). New cstate: current_term: 1 leader_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } health_report { overall_health: HEALTHY } } }
I20260929 15:52:04.032759 24241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:04.027402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55764 (local address 127.15.196.194:38361)
0929 15:52:04.027609 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:04.027615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:04.027633 (+ 18us) server_negotiation.cc:407] Connection header received
0929 15:52:04.027867 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:04.027871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:04.027945 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:04.028100 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:52:04.028445 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.029179 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.030522 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.030925 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.031122 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.031632 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:52:04.032177 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:52:04.032338 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:04.032394 (+ 56us) server_negotiation.cc:299] Negotiation successful
0929 15:52:04.032479 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 15:52:04.041109 23895 raft_consensus.cc:1261] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Refusing update from remote peer af29a2e9090c4a5fb36fe839cdd4646b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:52:04.041109 24175 raft_consensus.cc:1261] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Refusing update from remote peer af29a2e9090c4a5fb36fe839cdd4646b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:52:04.042068 24255 consensus_queue.cc:1035] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [LEADER]: Connected to new peer: Peer: permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:52:04.042429 24252 consensus_queue.cc:1035] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:52:04.057072 24260 mvcc.cc:205] Tried to move back new op lower bound from 7334695420061667328 to 7334695419962503168. Current Snapshot: MvccSnapshot[applied={T|T < 7334695420061667328}]
I20260929 15:52:04.082568 23733 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:52:04.082842 23733 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:52:04.086448 23733 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3: Sending DeleteTablet for 3 replicas of tablet 29fa3c20d67f45949b9d1634c2c17d67
I20260929 15:52:04.087787 24154 tablet_service.cc:1514] Processing DeleteTablet for tablet 29fa3c20d67f45949b9d1634c2c17d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:52:04 UTC) from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:56452
I20260929 15:52:04.087827 23871 tablet_service.cc:1514] Processing DeleteTablet for tablet 29fa3c20d67f45949b9d1634c2c17d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:52:04 UTC) from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:37970
I20260929 15:52:04.087982 24010 tablet_service.cc:1514] Processing DeleteTablet for tablet 29fa3c20d67f45949b9d1634c2c17d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:52:04 UTC) from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:55758
I20260929 15:52:04.090207 23733 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:52:04.092151 23733 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:52:04.104110 23870 tablet_service.cc:1467] Processing CreateTablet for tablet 7f4278d588ce4ecabb5550e61951feba (DEFAULT_TABLE table=test-table [id=8b53f62cc42f4adeabf320abdfb0201a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:52:04.104108 24153 tablet_service.cc:1467] Processing CreateTablet for tablet 7f4278d588ce4ecabb5550e61951feba (DEFAULT_TABLE table=test-table [id=8b53f62cc42f4adeabf320abdfb0201a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:52:04.104709 24153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f4278d588ce4ecabb5550e61951feba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:04.104720 23870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f4278d588ce4ecabb5550e61951feba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:04.108161 24009 tablet_service.cc:1467] Processing CreateTablet for tablet 7f4278d588ce4ecabb5550e61951feba (DEFAULT_TABLE table=test-table [id=8b53f62cc42f4adeabf320abdfb0201a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:52:04.108703 24009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f4278d588ce4ecabb5550e61951feba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:04.114188 24246 tablet_bootstrap.cc:492] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76: Bootstrap starting.
I20260929 15:52:04.117089 24246 tablet_bootstrap.cc:654] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:04.119324 24245 tablet_bootstrap.cc:492] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b: Bootstrap starting.
I20260929 15:52:04.121282 24245 tablet_bootstrap.cc:654] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:04.130445 24244 tablet_bootstrap.cc:492] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c: Bootstrap starting.
I20260929 15:52:04.132789 24244 tablet_bootstrap.cc:654] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:04.156777 24245 tablet_bootstrap.cc:492] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b: No bootstrap required, opened a new log
I20260929 15:52:04.157133 24245 ts_tablet_manager.cc:1397] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.010s
I20260929 15:52:04.158421 24245 raft_consensus.cc:348] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.158838 24245 raft_consensus.cc:374] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:04.159026 24245 raft_consensus.cc:729] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af29a2e9090c4a5fb36fe839cdd4646b, State: Initialized, Role: FOLLOWER
I20260929 15:52:04.160738 24244 tablet_bootstrap.cc:492] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c: No bootstrap required, opened a new log
I20260929 15:52:04.161028 24244 ts_tablet_manager.cc:1397] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c: Time spent bootstrapping tablet: real 0.031s user 0.008s sys 0.004s
I20260929 15:52:04.163834 24244 raft_consensus.cc:348] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.164265 24244 raft_consensus.cc:374] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:04.164441 24244 raft_consensus.cc:729] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 563a2b2e105b4346b04102d7f35be39c, State: Initialized, Role: FOLLOWER
I20260929 15:52:04.165344 24272 tablet_replica.cc:330] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b: stopping tablet replica
I20260929 15:52:04.166438 24272 raft_consensus.cc:2229] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:52:04.166833 24244 consensus_queue.cc:260] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.167987 24244 ts_tablet_manager.cc:1428] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260929 15:52:04.168646 24272 raft_consensus.cc:2258] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:52:04.170024 24271 tablet_replica.cc:330] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76: stopping tablet replica
I20260929 15:52:04.169203 24270 tablet_replica.cc:330] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c: stopping tablet replica
I20260929 15:52:04.167299 24245 consensus_queue.cc:260] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.170809 24270 raft_consensus.cc:2229] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:52:04.171221 24245 ts_tablet_manager.cc:1428] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260929 15:52:04.171847 24271 raft_consensus.cc:2229] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:52:04.172791 24246 tablet_bootstrap.cc:492] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76: No bootstrap required, opened a new log
I20260929 15:52:04.172947 24246 ts_tablet_manager.cc:1397] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76: Time spent bootstrapping tablet: real 0.059s user 0.012s sys 0.018s
I20260929 15:52:04.173362 24271 raft_consensus.cc:2258] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:52:04.174804 24270 raft_consensus.cc:2258] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:52:04.173805 24246 raft_consensus.cc:348] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.175020 24246 raft_consensus.cc:374] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:04.175128 24246 raft_consensus.cc:729] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72220c68c774184b366c2dacdca8f76, State: Initialized, Role: FOLLOWER
I20260929 15:52:04.175408 24246 consensus_queue.cc:260] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.176002 24246 ts_tablet_manager.cc:1428] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 15:52:04.176846 24272 ts_tablet_manager.cc:1905] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:52:04.178969 24270 ts_tablet_manager.cc:1905] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:52:04.179775 24271 ts_tablet_manager.cc:1905] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:52:04.184170 24272 ts_tablet_manager.cc:1918] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:52:04.184401 24272 log.cc:1196] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/29fa3c20d67f45949b9d1634c2c17d67
I20260929 15:52:04.185061 24272 ts_tablet_manager.cc:1939] T 29fa3c20d67f45949b9d1634c2c17d67 P af29a2e9090c4a5fb36fe839cdd4646b: Deleting consensus metadata
I20260929 15:52:04.186600 24271 ts_tablet_manager.cc:1918] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:52:04.186836 24271 log.cc:1196] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/29fa3c20d67f45949b9d1634c2c17d67
I20260929 15:52:04.186967 23720 catalog_manager.cc:4841] TS af29a2e9090c4a5fb36fe839cdd4646b (127.15.196.194:38361): tablet 29fa3c20d67f45949b9d1634c2c17d67 (table test-table [id=36850c23cab34fd4be74daedf8129fdc]) successfully deleted
I20260929 15:52:04.187500 24271 ts_tablet_manager.cc:1939] T 29fa3c20d67f45949b9d1634c2c17d67 P d72220c68c774184b366c2dacdca8f76: Deleting consensus metadata
I20260929 15:52:04.188902 23719 catalog_manager.cc:4841] TS d72220c68c774184b366c2dacdca8f76 (127.15.196.193:33007): tablet 29fa3c20d67f45949b9d1634c2c17d67 (table test-table [id=36850c23cab34fd4be74daedf8129fdc]) successfully deleted
I20260929 15:52:04.190799 24270 ts_tablet_manager.cc:1918] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:52:04.191011 24270 log.cc:1196] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/29fa3c20d67f45949b9d1634c2c17d67
I20260929 15:52:04.191588 24270 ts_tablet_manager.cc:1939] T 29fa3c20d67f45949b9d1634c2c17d67 P 563a2b2e105b4346b04102d7f35be39c: Deleting consensus metadata
I20260929 15:52:04.193296 23719 catalog_manager.cc:4841] TS 563a2b2e105b4346b04102d7f35be39c (127.15.196.195:42727): tablet 29fa3c20d67f45949b9d1634c2c17d67 (table test-table [id=36850c23cab34fd4be74daedf8129fdc]) successfully deleted
W20260929 15:52:04.217893 23942 tablet.cc:2369] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:52:04.238720 24255 raft_consensus.cc:482] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:52:04.239048 24255 raft_consensus.cc:504] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.239928 24255 leader_election.cc:288] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d72220c68c774184b366c2dacdca8f76 (127.15.196.193:33007), 563a2b2e105b4346b04102d7f35be39c (127.15.196.195:42727)
I20260929 15:52:04.240799 23895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f4278d588ce4ecabb5550e61951feba" candidate_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72220c68c774184b366c2dacdca8f76" is_pre_election: true
I20260929 15:52:04.240898 24175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f4278d588ce4ecabb5550e61951feba" candidate_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "563a2b2e105b4346b04102d7f35be39c" is_pre_election: true
I20260929 15:52:04.241068 23895 raft_consensus.cc:2454] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af29a2e9090c4a5fb36fe839cdd4646b in term 0.
I20260929 15:52:04.241163 24175 raft_consensus.cc:2454] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af29a2e9090c4a5fb36fe839cdd4646b in term 0.
I20260929 15:52:04.241557 23962 leader_election.cc:302] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af29a2e9090c4a5fb36fe839cdd4646b, d72220c68c774184b366c2dacdca8f76; no voters:
I20260929 15:52:04.241863 24255 raft_consensus.cc:2781] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:52:04.241973 24255 raft_consensus.cc:482] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:52:04.242100 24255 raft_consensus.cc:3037] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:04.244402 24255 raft_consensus.cc:504] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.244992 24255 leader_election.cc:288] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [CANDIDATE]: Term 1 election: Requested vote from peers d72220c68c774184b366c2dacdca8f76 (127.15.196.193:33007), 563a2b2e105b4346b04102d7f35be39c (127.15.196.195:42727)
I20260929 15:52:04.245446 23895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f4278d588ce4ecabb5550e61951feba" candidate_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72220c68c774184b366c2dacdca8f76"
I20260929 15:52:04.245527 24175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f4278d588ce4ecabb5550e61951feba" candidate_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "563a2b2e105b4346b04102d7f35be39c"
I20260929 15:52:04.245644 23895 raft_consensus.cc:3037] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:04.245730 24175 raft_consensus.cc:3037] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:04.247710 23895 raft_consensus.cc:2454] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af29a2e9090c4a5fb36fe839cdd4646b in term 1.
I20260929 15:52:04.248162 23962 leader_election.cc:302] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af29a2e9090c4a5fb36fe839cdd4646b, d72220c68c774184b366c2dacdca8f76; no voters:
I20260929 15:52:04.248423 24255 raft_consensus.cc:2781] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:52:04.248656 24175 raft_consensus.cc:2454] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af29a2e9090c4a5fb36fe839cdd4646b in term 1.
I20260929 15:52:04.248631 24255 raft_consensus.cc:686] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [term 1 LEADER]: Becoming Leader. State: Replica: af29a2e9090c4a5fb36fe839cdd4646b, State: Running, Role: LEADER
I20260929 15:52:04.249178 24255 consensus_queue.cc:237] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.252005 23734 catalog_manager.cc:5495] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b reported cstate change: term changed from 0 to 1, leader changed from <none> to af29a2e9090c4a5fb36fe839cdd4646b (127.15.196.194). New cstate: current_term: 1 leader_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } health_report { overall_health: UNKNOWN } } }
Sep 29 15:52:04 dist-test-slave-c51v krb5kdc[23689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697121, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.254@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM
I20260929 15:52:04.312318 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:04.293868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:41821 (local address 127.0.0.1:48268)
0929 15:52:04.294454 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:04.294469 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:04.294650 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:04.294962 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:04.294966 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:04.295758 (+ 792us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:04.296009 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:04.296019 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.297016 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.297021 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:04.297977 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:04.297988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.298671 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.298677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:04.298838 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.303891 (+ 5053us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:04.303909 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:04.306297 (+ 2388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:04.309306 (+ 3009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:04.309321 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:04.309329 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:04.309813 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:04.310194 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:04.310223 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:04.310230 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:04.310408 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:04.311000 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:04.311009 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:04.311237 (+ 228us) client_negotiation.cc:769] Sending connection context
0929 15:52:04.311445 (+ 208us) client_negotiation.cc:240] Negotiation successful
0929 15:52:04.311773 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":324,"threads_started":1}
I20260929 15:52:04.312337 24107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:04.293809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48268 (local address 127.15.196.254:41821)
0929 15:52:04.294023 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:04.294027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:04.294567 (+ 540us) server_negotiation.cc:407] Connection header received
0929 15:52:04.294780 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:04.294789 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:04.294841 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:04.294941 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:04.296157 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.296882 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.298115 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.298498 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.298633 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.306496 (+ 7863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:04.306501 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:04.306504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:04.306538 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:04.309171 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:04.309924 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:04.309932 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:04.309935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:04.310049 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:04.310524 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:04.310528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:04.310530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:04.310871 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:04.310978 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:04.311538 (+ 560us) server_negotiation.cc:299] Negotiation successful
0929 15:52:04.311971 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":151,"server-negotiator.queue_time_us":96}
W20260929 15:52:04.313175 24221 tablet.cc:2369] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:52:04.317847 23734 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:48268:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 15:52:04.318881 23734 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:52:04.326061 24010 tablet_service.cc:1467] Processing CreateTablet for tablet e1d86fbee64446519ca85aeb0571d7bd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b513fef9d9c74d68a50f15a0cd287485]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 15:52:04.326061 23871 tablet_service.cc:1467] Processing CreateTablet for tablet e1d86fbee64446519ca85aeb0571d7bd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b513fef9d9c74d68a50f15a0cd287485]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 15:52:04.326663 23871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1d86fbee64446519ca85aeb0571d7bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:04.326668 24010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1d86fbee64446519ca85aeb0571d7bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:04.332388 24154 tablet_service.cc:1467] Processing CreateTablet for tablet e1d86fbee64446519ca85aeb0571d7bd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b513fef9d9c74d68a50f15a0cd287485]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 15:52:04.332943 24154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1d86fbee64446519ca85aeb0571d7bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:04.333779 24246 tablet_bootstrap.cc:492] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76: Bootstrap starting.
I20260929 15:52:04.335425 24246 tablet_bootstrap.cc:654] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:04.336064 24245 tablet_bootstrap.cc:492] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b: Bootstrap starting.
I20260929 15:52:04.337718 24245 tablet_bootstrap.cc:654] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:04.341682 24244 tablet_bootstrap.cc:492] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c: Bootstrap starting.
I20260929 15:52:04.343547 24244 tablet_bootstrap.cc:654] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:04.345955 24246 tablet_bootstrap.cc:492] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76: No bootstrap required, opened a new log
I20260929 15:52:04.346103 24244 tablet_bootstrap.cc:492] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c: No bootstrap required, opened a new log
I20260929 15:52:04.346150 24246 ts_tablet_manager.cc:1397] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.008s
I20260929 15:52:04.346226 24245 tablet_bootstrap.cc:492] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b: No bootstrap required, opened a new log
I20260929 15:52:04.346275 24244 ts_tablet_manager.cc:1397] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260929 15:52:04.346421 24245 ts_tablet_manager.cc:1397] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260929 15:52:04.346969 24246 raft_consensus.cc:348] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.347047 24244 raft_consensus.cc:348] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.347273 24246 raft_consensus.cc:374] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:04.347189 24245 raft_consensus.cc:348] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.347407 24246 raft_consensus.cc:729] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72220c68c774184b366c2dacdca8f76, State: Initialized, Role: FOLLOWER
I20260929 15:52:04.347448 24245 raft_consensus.cc:374] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:04.347545 24245 raft_consensus.cc:729] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af29a2e9090c4a5fb36fe839cdd4646b, State: Initialized, Role: FOLLOWER
I20260929 15:52:04.347837 24244 raft_consensus.cc:374] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:04.347944 24244 raft_consensus.cc:729] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 563a2b2e105b4346b04102d7f35be39c, State: Initialized, Role: FOLLOWER
I20260929 15:52:04.347800 24246 consensus_queue.cc:260] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.348176 24244 consensus_queue.cc:260] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.348430 24245 consensus_queue.cc:260] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.348613 24246 ts_tablet_manager.cc:1428] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 15:52:04.349042 24245 ts_tablet_manager.cc:1428] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 15:52:04.348922 24255 tablet_replica.cc:439] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } }
I20260929 15:52:04.349182 24255 tablet_replica.cc:442] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:52:04.349354 24244 ts_tablet_manager.cc:1428] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260929 15:52:04.350116 24250 tablet_replica.cc:439] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } }
I20260929 15:52:04.350414 24250 tablet_replica.cc:442] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:52:04.350701 24251 tablet_replica.cc:439] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } }
I20260929 15:52:04.351016 24251 tablet_replica.cc:442] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:52:04.506907 24250 raft_consensus.cc:482] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:52:04.507125 24250 raft_consensus.cc:504] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.508677 24250 leader_election.cc:288] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d72220c68c774184b366c2dacdca8f76 (127.15.196.193:33007), af29a2e9090c4a5fb36fe839cdd4646b (127.15.196.194:38361)
I20260929 15:52:04.519979 24241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:04.509044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:48267 (local address 127.15.196.194:38361)
0929 15:52:04.509562 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:04.509568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:04.509592 (+ 24us) server_negotiation.cc:407] Connection header received
0929 15:52:04.509883 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:04.509892 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:04.510008 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:04.510081 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:52:04.511268 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.512851 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.516364 (+ 3513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.518229 (+ 1865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.518402 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.518566 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:04.519329 (+ 763us) server_negotiation.cc:299] Negotiation successful
0929 15:52:04.519570 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373}
I20260929 15:52:04.520740 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:04.508906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:38361 (local address 127.15.196.195:48267)
0929 15:52:04.509447 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:04.509485 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:04.509668 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:04.510338 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:04.510345 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:04.510379 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:52:04.510667 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:04.510683 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.513142 (+ 2459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.513150 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:04.516140 (+ 2990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:04.516160 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.518766 (+ 2606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.518775 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:04.519042 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.519098 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 15:52:04.519880 (+ 782us) client_negotiation.cc:240] Negotiation successful
0929 15:52:04.520375 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":316,"threads_started":1}
I20260929 15:52:04.521935 24034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d86fbee64446519ca85aeb0571d7bd" candidate_uuid: "563a2b2e105b4346b04102d7f35be39c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" is_pre_election: true
I20260929 15:52:04.522385 24034 raft_consensus.cc:2454] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 563a2b2e105b4346b04102d7f35be39c in term 0.
I20260929 15:52:04.522987 24102 leader_election.cc:302] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 563a2b2e105b4346b04102d7f35be39c, af29a2e9090c4a5fb36fe839cdd4646b; no voters:
I20260929 15:52:04.523522 24250 raft_consensus.cc:2781] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:52:04.523660 24250 raft_consensus.cc:482] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:52:04.523798 24250 raft_consensus.cc:3037] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:04.526682 24250 raft_consensus.cc:504] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.528256 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:04.509241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:44087 (local address 127.15.196.193:33007)
0929 15:52:04.519674 (+ 10433us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:04.519680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:04.520789 (+ 1109us) server_negotiation.cc:407] Connection header received
0929 15:52:04.520988 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:04.520992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:04.521087 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:04.521192 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:52:04.521817 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.522954 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.525545 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.526855 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.527039 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.527114 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:04.527770 (+ 656us) server_negotiation.cc:299] Negotiation successful
0929 15:52:04.527910 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10306,"thread_start_us":10179,"threads_started":1}
I20260929 15:52:04.528270 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:04.508909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:33007 (local address 127.15.196.195:44087)
0929 15:52:04.520682 (+ 11773us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:04.520699 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:04.520850 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:04.521288 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:04.521293 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:04.521319 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:52:04.521630 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:04.521641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.523120 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.523126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:04.525353 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:04.525368 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.527094 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.527101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:04.527581 (+ 480us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.527619 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 15:52:04.527730 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 15:52:04.527866 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11641,"mutex_wait_us":171,"thread_start_us":11289,"threads_started":1}
I20260929 15:52:04.528641 24250 leader_election.cc:288] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [CANDIDATE]: Term 1 election: Requested vote from peers d72220c68c774184b366c2dacdca8f76 (127.15.196.193:33007), af29a2e9090c4a5fb36fe839cdd4646b (127.15.196.194:38361)
I20260929 15:52:04.528777 24034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d86fbee64446519ca85aeb0571d7bd" candidate_uuid: "563a2b2e105b4346b04102d7f35be39c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af29a2e9090c4a5fb36fe839cdd4646b"
I20260929 15:52:04.529006 24034 raft_consensus.cc:3037] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:04.529450 23895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d86fbee64446519ca85aeb0571d7bd" candidate_uuid: "563a2b2e105b4346b04102d7f35be39c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72220c68c774184b366c2dacdca8f76" is_pre_election: true
I20260929 15:52:04.529633 23894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d86fbee64446519ca85aeb0571d7bd" candidate_uuid: "563a2b2e105b4346b04102d7f35be39c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72220c68c774184b366c2dacdca8f76"
I20260929 15:52:04.529743 23895 raft_consensus.cc:2454] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 563a2b2e105b4346b04102d7f35be39c in term 0.
I20260929 15:52:04.529866 23894 raft_consensus.cc:3037] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:04.532096 23894 raft_consensus.cc:2454] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 563a2b2e105b4346b04102d7f35be39c in term 1.
I20260929 15:52:04.532213 24034 raft_consensus.cc:2454] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 563a2b2e105b4346b04102d7f35be39c in term 1.
I20260929 15:52:04.532509 24101 leader_election.cc:302] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 563a2b2e105b4346b04102d7f35be39c, d72220c68c774184b366c2dacdca8f76; no voters:
I20260929 15:52:04.533003 24250 raft_consensus.cc:2781] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:52:04.533445 24250 raft_consensus.cc:686] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [term 1 LEADER]: Becoming Leader. State: Replica: 563a2b2e105b4346b04102d7f35be39c, State: Running, Role: LEADER
I20260929 15:52:04.533874 24250 consensus_queue.cc:237] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.535837 24289 tablet_replica.cc:439] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c: TxnStatusTablet state changed. Reason: New leader 563a2b2e105b4346b04102d7f35be39c. Latest consensus state: current_term: 1 leader_uuid: "563a2b2e105b4346b04102d7f35be39c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } }
I20260929 15:52:04.536194 24289 tablet_replica.cc:442] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c: This TxnStatusTablet replica's current role is: LEADER
I20260929 15:52:04.537089 24291 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 15:52:04.537312 24291 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 15:52:04.537707 23734 catalog_manager.cc:5495] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c reported cstate change: term changed from 0 to 1, leader changed from <none> to 563a2b2e105b4346b04102d7f35be39c (127.15.196.195). New cstate: current_term: 1 leader_uuid: "563a2b2e105b4346b04102d7f35be39c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } health_report { overall_health: HEALTHY } } }
I20260929 15:52:04.587666 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:04.581795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:42727 (local address 127.0.0.1:56462)
0929 15:52:04.582025 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:04.582050 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:04.582209 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:04.582764 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:04.582769 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:04.582782 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 15:52:04.583027 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:04.583042 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.584224 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.584229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:04.585221 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:04.585232 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.585963 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.585969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:04.586136 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.586188 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 15:52:04.587069 (+ 881us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 15:52:04.587091 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 15:52:04.587175 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 15:52:04.587347 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 15:52:04.587708 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:04.581851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56462 (local address 127.15.196.195:42727)
0929 15:52:04.582401 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:04.582407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:04.582422 (+ 15us) server_negotiation.cc:407] Connection header received
0929 15:52:04.582498 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:04.582505 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:04.582579 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:04.582741 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:52:04.583201 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.584082 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.585367 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.585804 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.585954 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.586387 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:52:04.586954 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:52:04.587112 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:04.587211 (+ 99us) server_negotiation.cc:299] Negotiation successful
0929 15:52:04.587328 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":249,"threads_started":1}
I20260929 15:52:04.592672 23894 raft_consensus.cc:1261] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Refusing update from remote peer 563a2b2e105b4346b04102d7f35be39c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:52:04.592803 24034 raft_consensus.cc:1261] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b [term 1 FOLLOWER]: Refusing update from remote peer 563a2b2e105b4346b04102d7f35be39c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:52:04.593326 24250 consensus_queue.cc:1035] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:52:04.593636 24289 consensus_queue.cc:1035] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c [LEADER]: Connected to new peer: Peer: permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:52:04.600263 24251 tablet_replica.cc:439] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76: TxnStatusTablet state changed. Reason: New leader 563a2b2e105b4346b04102d7f35be39c. Latest consensus state: current_term: 1 leader_uuid: "563a2b2e105b4346b04102d7f35be39c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } }
I20260929 15:52:04.600618 24251 tablet_replica.cc:442] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:52:04.603474 24255 tablet_replica.cc:439] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b: TxnStatusTablet state changed. Reason: New leader 563a2b2e105b4346b04102d7f35be39c. Latest consensus state: current_term: 1 leader_uuid: "563a2b2e105b4346b04102d7f35be39c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } }
I20260929 15:52:04.603771 24255 tablet_replica.cc:442] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:52:04.605413 24295 mvcc.cc:205] Tried to move back new op lower bound from 7334695422324088832 to 7334695422094512128. Current Snapshot: MvccSnapshot[applied={T|T < 7334695422324088832}]
I20260929 15:52:04.614111 24255 tablet_replica.cc:439] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "563a2b2e105b4346b04102d7f35be39c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } }
I20260929 15:52:04.614399 24255 tablet_replica.cc:442] T e1d86fbee64446519ca85aeb0571d7bd P af29a2e9090c4a5fb36fe839cdd4646b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:52:04.616888 24289 tablet_replica.cc:439] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "563a2b2e105b4346b04102d7f35be39c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } }
I20260929 15:52:04.617167 24289 tablet_replica.cc:442] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c: This TxnStatusTablet replica's current role is: LEADER
I20260929 15:52:04.617524 24251 tablet_replica.cc:439] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "563a2b2e105b4346b04102d7f35be39c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } }
I20260929 15:52:04.617766 24251 tablet_replica.cc:442] T e1d86fbee64446519ca85aeb0571d7bd P d72220c68c774184b366c2dacdca8f76: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 15:52:04.618353 24250 tablet_replica.cc:439] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "563a2b2e105b4346b04102d7f35be39c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } } }
I20260929 15:52:04.618620 24250 tablet_replica.cc:442] T e1d86fbee64446519ca85aeb0571d7bd P 563a2b2e105b4346b04102d7f35be39c: This TxnStatusTablet replica's current role is: LEADER
I20260929 15:52:04.639992 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:04.634694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:45157 (local address 127.15.196.195:42727)
0929 15:52:04.634889 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:04.634897 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:04.635270 (+ 373us) server_negotiation.cc:407] Connection header received
0929 15:52:04.635521 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:04.635525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:04.635593 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:04.635692 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:52:04.636149 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.636873 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.638187 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.638644 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.638795 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.639225 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:52:04.639385 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:52:04.639482 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:04.639625 (+ 143us) server_negotiation.cc:299] Negotiation successful
0929 15:52:04.639704 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 15:52:04.640159 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:04.634588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:42727 (local address 127.15.196.194:45157)
0929 15:52:04.635162 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:04.635180 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:04.635367 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:04.635706 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:04.635711 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:04.635722 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 15:52:04.636002 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:04.636011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.637009 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.637017 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:04.638052 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:04.638063 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.638857 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.638870 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:04.638993 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.639047 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 15:52:04.639495 (+ 448us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 15:52:04.639508 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 15:52:04.639604 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 15:52:04.639755 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":309,"threads_started":1}
I20260929 15:52:04.649688 24175 raft_consensus.cc:1261] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Refusing update from remote peer af29a2e9090c4a5fb36fe839cdd4646b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:52:04.649797 23894 raft_consensus.cc:1261] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Refusing update from remote peer af29a2e9090c4a5fb36fe839cdd4646b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:52:04.650214 24267 consensus_queue.cc:1035] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [LEADER]: Connected to new peer: Peer: permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:52:04.650550 24255 consensus_queue.cc:1035] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:52:04.658080 24297 mvcc.cc:205] Tried to move back new op lower bound from 7334695422560219136 to 7334695420926963712. Current Snapshot: MvccSnapshot[applied={T|T < 7334695422560219136}]
I20260929 15:52:04.689607 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:04.679198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:38361 (local address 127.15.196.195:48003)
0929 15:52:04.683818 (+ 4620us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:04.683835 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:04.684008 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:04.684525 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:04.684528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:04.684538 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 15:52:04.684877 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:04.684886 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.686110 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.686116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:04.687414 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:04.687428 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.688367 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.688380 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:04.688502 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.688558 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 15:52:04.689052 (+ 494us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 15:52:04.689074 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 15:52:04.689155 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 15:52:04.689302 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4423,"thread_start_us":4295,"threads_started":1}
I20260929 15:52:04.689599 24241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:04.679245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:48003 (local address 127.15.196.194:38361)
0929 15:52:04.679479 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:04.679489 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:04.683924 (+ 4435us) server_negotiation.cc:407] Connection header received
0929 15:52:04.684155 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:04.684161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:04.684265 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:04.684355 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:52:04.685044 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.685948 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.687604 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.688058 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.688159 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.688756 (+ 597us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:52:04.688916 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:52:04.689004 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:04.689215 (+ 211us) server_negotiation.cc:299] Negotiation successful
0929 15:52:04.689303 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 15:52:04.736306 23734 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:52:04.736558 23734 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:52:04.739701 23734 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3: Sending DeleteTablet for 3 replicas of tablet 7f4278d588ce4ecabb5550e61951feba
I20260929 15:52:04.740491 23871 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f4278d588ce4ecabb5550e61951feba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:52:04 UTC) from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:37970
I20260929 15:52:04.741150 24010 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f4278d588ce4ecabb5550e61951feba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:52:04 UTC) from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:55758
I20260929 15:52:04.747689 24154 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f4278d588ce4ecabb5550e61951feba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:52:04 UTC) from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:56452
I20260929 15:52:04.751662 24313 tablet_replica.cc:330] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76: stopping tablet replica
I20260929 15:52:04.751942 24313 raft_consensus.cc:2229] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:52:04.752221 24313 raft_consensus.cc:2258] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:52:04.753415 24313 ts_tablet_manager.cc:1905] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:52:04.759686 24314 tablet_replica.cc:330] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b: stopping tablet replica
I20260929 15:52:04.759747 24315 tablet_replica.cc:330] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c: stopping tablet replica
I20260929 15:52:04.759938 24314 raft_consensus.cc:2229] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:52:04.759958 24315 raft_consensus.cc:2229] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:52:04.760394 24315 raft_consensus.cc:2258] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:52:04.760399 24314 raft_consensus.cc:2258] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:52:04.762113 24315 ts_tablet_manager.cc:1905] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:52:04.764096 24314 ts_tablet_manager.cc:1905] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:52:04.770978 24315 ts_tablet_manager.cc:1918] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 15:52:04.771138 24315 log.cc:1196] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/7f4278d588ce4ecabb5550e61951feba
I20260929 15:52:04.771672 24315 ts_tablet_manager.cc:1939] T 7f4278d588ce4ecabb5550e61951feba P 563a2b2e105b4346b04102d7f35be39c: Deleting consensus metadata
I20260929 15:52:04.772912 23719 catalog_manager.cc:4841] TS 563a2b2e105b4346b04102d7f35be39c (127.15.196.195:42727): tablet 7f4278d588ce4ecabb5550e61951feba (table test-table [id=8b53f62cc42f4adeabf320abdfb0201a]) successfully deleted
I20260929 15:52:04.774372 24313 ts_tablet_manager.cc:1918] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 15:52:04.774533 24313 log.cc:1196] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/7f4278d588ce4ecabb5550e61951feba
I20260929 15:52:04.774953 24313 ts_tablet_manager.cc:1939] T 7f4278d588ce4ecabb5550e61951feba P d72220c68c774184b366c2dacdca8f76: Deleting consensus metadata
I20260929 15:52:04.776109 23719 catalog_manager.cc:4841] TS d72220c68c774184b366c2dacdca8f76 (127.15.196.193:33007): tablet 7f4278d588ce4ecabb5550e61951feba (table test-table [id=8b53f62cc42f4adeabf320abdfb0201a]) successfully deleted
I20260929 15:52:04.779629 24314 ts_tablet_manager.cc:1918] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 15:52:04.779801 24314 log.cc:1196] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/7f4278d588ce4ecabb5550e61951feba
I20260929 15:52:04.780201 24314 ts_tablet_manager.cc:1939] T 7f4278d588ce4ecabb5550e61951feba P af29a2e9090c4a5fb36fe839cdd4646b: Deleting consensus metadata
I20260929 15:52:04.781126 23720 catalog_manager.cc:4841] TS af29a2e9090c4a5fb36fe839cdd4646b (127.15.196.194:38361): tablet 7f4278d588ce4ecabb5550e61951feba (table test-table [id=8b53f62cc42f4adeabf320abdfb0201a]) successfully deleted
Sep 29 15:52:04 dist-test-slave-c51v krb5kdc[23689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 15:52:04 dist-test-slave-c51v krb5kdc[23689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:04 dist-test-slave-c51v krb5kdc[23689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 15:52:04.815811 24107 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:04.804602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48270 (local address 127.15.196.254:41821)
0929 15:52:04.804832 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:04.804841 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:04.805357 (+ 516us) server_negotiation.cc:407] Connection header received
0929 15:52:04.805605 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:04.805610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:04.805674 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:04.805783 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:04.807180 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.808220 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.809595 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.810079 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.810210 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.815536 (+ 5326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48270: BlockingRecv error: recv got EOF from 127.0.0.1:48270 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 29 15:52:04 dist-test-slave-c51v krb5kdc[23689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM
I20260929 15:52:04.835119 24107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:04.819245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48278 (local address 127.15.196.254:41821)
0929 15:52:04.819449 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:04.819454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:04.820034 (+ 580us) server_negotiation.cc:407] Connection header received
0929 15:52:04.820246 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:04.820250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:04.820321 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:04.820435 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:04.821944 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.822755 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.824096 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:04.824517 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:04.824656 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:04.828935 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:04.828945 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:04.828952 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:04.828999 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:04.832445 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:04.833130 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:04.833135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:04.833139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:04.833266 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:04.833670 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:04.833674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:04.833678 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:04.833959 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:04.834274 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:04.834535 (+ 261us) server_negotiation.cc:299] Negotiation successful
0929 15:52:04.834755 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 15:52:04.839008 23734 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:52:04.839759 23734 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:52:04.847107 24010 tablet_service.cc:1467] Processing CreateTablet for tablet 4b9f404daed84198a868845d13ba7c59 (DEFAULT_TABLE table=test-table [id=f94160e05276421c8a2161cb5f0256b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:52:04.847355 23871 tablet_service.cc:1467] Processing CreateTablet for tablet 4b9f404daed84198a868845d13ba7c59 (DEFAULT_TABLE table=test-table [id=f94160e05276421c8a2161cb5f0256b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:52:04.847617 24010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b9f404daed84198a868845d13ba7c59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:04.847743 23871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b9f404daed84198a868845d13ba7c59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:04.854802 24154 tablet_service.cc:1467] Processing CreateTablet for tablet 4b9f404daed84198a868845d13ba7c59 (DEFAULT_TABLE table=test-table [id=f94160e05276421c8a2161cb5f0256b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:52:04.855405 24154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b9f404daed84198a868845d13ba7c59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:04.859817 24329 tablet_bootstrap.cc:492] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76: Bootstrap starting.
I20260929 15:52:04.860062 24328 tablet_bootstrap.cc:492] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b: Bootstrap starting.
I20260929 15:52:04.861697 24329 tablet_bootstrap.cc:654] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:04.862531 24328 tablet_bootstrap.cc:654] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:04.862854 24330 tablet_bootstrap.cc:492] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c: Bootstrap starting.
I20260929 15:52:04.866456 24330 tablet_bootstrap.cc:654] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:04.868228 24328 tablet_bootstrap.cc:492] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b: No bootstrap required, opened a new log
I20260929 15:52:04.868403 24328 ts_tablet_manager.cc:1397] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 15:52:04.869217 24328 raft_consensus.cc:348] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.869513 24328 raft_consensus.cc:374] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:04.869606 24328 raft_consensus.cc:729] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af29a2e9090c4a5fb36fe839cdd4646b, State: Initialized, Role: FOLLOWER
I20260929 15:52:04.870020 24328 consensus_queue.cc:260] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.874624 24328 ts_tablet_manager.cc:1428] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260929 15:52:04.879065 24329 tablet_bootstrap.cc:492] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76: No bootstrap required, opened a new log
I20260929 15:52:04.879240 24329 ts_tablet_manager.cc:1397] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76: Time spent bootstrapping tablet: real 0.020s user 0.019s sys 0.000s
I20260929 15:52:04.879716 24330 tablet_bootstrap.cc:492] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c: No bootstrap required, opened a new log
I20260929 15:52:04.879884 24330 ts_tablet_manager.cc:1397] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.006s
I20260929 15:52:04.880133 24329 raft_consensus.cc:348] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.880407 24329 raft_consensus.cc:374] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:04.880509 24329 raft_consensus.cc:729] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72220c68c774184b366c2dacdca8f76, State: Initialized, Role: FOLLOWER
I20260929 15:52:04.880782 24329 consensus_queue.cc:260] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.880745 24330 raft_consensus.cc:348] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.881023 24330 raft_consensus.cc:374] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:04.881129 24330 raft_consensus.cc:729] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 563a2b2e105b4346b04102d7f35be39c, State: Initialized, Role: FOLLOWER
I20260929 15:52:04.881361 24329 ts_tablet_manager.cc:1428] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 15:52:04.881404 24330 consensus_queue.cc:260] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.882067 24330 ts_tablet_manager.cc:1428] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 15:52:04.995831 24312 raft_consensus.cc:482] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:52:04.996120 24312 raft_consensus.cc:504] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:04.997026 24312 leader_election.cc:288] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d72220c68c774184b366c2dacdca8f76 (127.15.196.193:33007), 563a2b2e105b4346b04102d7f35be39c (127.15.196.195:42727)
I20260929 15:52:04.997656 23894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b9f404daed84198a868845d13ba7c59" candidate_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72220c68c774184b366c2dacdca8f76" is_pre_election: true
I20260929 15:52:04.997730 24175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b9f404daed84198a868845d13ba7c59" candidate_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "563a2b2e105b4346b04102d7f35be39c" is_pre_election: true
I20260929 15:52:04.997954 23894 raft_consensus.cc:2454] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af29a2e9090c4a5fb36fe839cdd4646b in term 0.
I20260929 15:52:04.998000 24175 raft_consensus.cc:2454] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af29a2e9090c4a5fb36fe839cdd4646b in term 0.
I20260929 15:52:04.998389 23962 leader_election.cc:302] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af29a2e9090c4a5fb36fe839cdd4646b, d72220c68c774184b366c2dacdca8f76; no voters:
I20260929 15:52:04.998678 24312 raft_consensus.cc:2781] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:52:04.998796 24312 raft_consensus.cc:482] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:52:04.998881 24312 raft_consensus.cc:3037] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:05.001994 24312 raft_consensus.cc:504] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:05.002648 24312 leader_election.cc:288] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [CANDIDATE]: Term 1 election: Requested vote from peers d72220c68c774184b366c2dacdca8f76 (127.15.196.193:33007), 563a2b2e105b4346b04102d7f35be39c (127.15.196.195:42727)
I20260929 15:52:05.003221 23894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b9f404daed84198a868845d13ba7c59" candidate_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72220c68c774184b366c2dacdca8f76"
I20260929 15:52:05.003324 24175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b9f404daed84198a868845d13ba7c59" candidate_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "563a2b2e105b4346b04102d7f35be39c"
I20260929 15:52:05.003499 23894 raft_consensus.cc:3037] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:05.003544 24175 raft_consensus.cc:3037] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:05.006608 23894 raft_consensus.cc:2454] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af29a2e9090c4a5fb36fe839cdd4646b in term 1.
I20260929 15:52:05.006631 24175 raft_consensus.cc:2454] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af29a2e9090c4a5fb36fe839cdd4646b in term 1.
I20260929 15:52:05.007141 23962 leader_election.cc:302] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 563a2b2e105b4346b04102d7f35be39c, af29a2e9090c4a5fb36fe839cdd4646b; no voters:
I20260929 15:52:05.007531 24312 raft_consensus.cc:2781] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:52:05.007721 24312 raft_consensus.cc:686] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [term 1 LEADER]: Becoming Leader. State: Replica: af29a2e9090c4a5fb36fe839cdd4646b, State: Running, Role: LEADER
I20260929 15:52:05.008037 24312 consensus_queue.cc:237] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:05.011049 23734 catalog_manager.cc:5495] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b reported cstate change: term changed from 0 to 1, leader changed from <none> to af29a2e9090c4a5fb36fe839cdd4646b (127.15.196.194). New cstate: current_term: 1 leader_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } health_report { overall_health: UNKNOWN } } }
I20260929 15:52:05.057942 24241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:05.053502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55772 (local address 127.15.196.194:38361)
0929 15:52:05.053712 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:05.053717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:05.053736 (+ 19us) server_negotiation.cc:407] Connection header received
0929 15:52:05.053861 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:05.053865 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:05.053932 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:05.054020 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:52:05.054328 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:05.055085 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:05.056425 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:05.056843 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:05.056941 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:05.057300 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:52:05.057437 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:52:05.057506 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:05.057653 (+ 147us) server_negotiation.cc:299] Negotiation successful
0929 15:52:05.057717 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 15:52:05.060564 24175 raft_consensus.cc:1261] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Refusing update from remote peer af29a2e9090c4a5fb36fe839cdd4646b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:52:05.060611 23894 raft_consensus.cc:1261] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Refusing update from remote peer af29a2e9090c4a5fb36fe839cdd4646b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:52:05.061283 24255 consensus_queue.cc:1035] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [LEADER]: Connected to new peer: Peer: permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:52:05.061546 24312 consensus_queue.cc:1035] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:52:05.073499 23733 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:52:05.073797 23733 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:52:05.077827 23733 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3: Sending DeleteTablet for 3 replicas of tablet 4b9f404daed84198a868845d13ba7c59
I20260929 15:52:05.078821 24010 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b9f404daed84198a868845d13ba7c59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:52:05 UTC) from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:55758
I20260929 15:52:05.079090 24314 tablet_replica.cc:330] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b: stopping tablet replica
I20260929 15:52:05.079149 24154 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b9f404daed84198a868845d13ba7c59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:52:05 UTC) from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:56452
I20260929 15:52:05.079329 24314 raft_consensus.cc:2229] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:52:05.079447 24315 tablet_replica.cc:330] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c: stopping tablet replica
I20260929 15:52:05.079654 24315 raft_consensus.cc:2229] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:52:05.079663 24314 raft_consensus.cc:2258] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:52:05.079883 24315 raft_consensus.cc:2258] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:52:05.080071 23871 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b9f404daed84198a868845d13ba7c59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:52:05 UTC) from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:37970
I20260929 15:52:05.080240 23870 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b9f404daed84198a868845d13ba7c59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:52:05 UTC) from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:37970
I20260929 15:52:05.080332 24313 tablet_replica.cc:330] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76: stopping tablet replica
I20260929 15:52:05.080498 24313 raft_consensus.cc:2229] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:52:05.080722 24313 raft_consensus.cc:2258] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:52:05.080528 23733 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:52:05.081198 23733 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:52:05.081634 24313 ts_tablet_manager.cc:1905] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:52:05.081776 24314 ts_tablet_manager.cc:1905] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:52:05.086618 24315 ts_tablet_manager.cc:1905] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:52:05.088330 24314 ts_tablet_manager.cc:1918] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:52:05.088482 24314 log.cc:1196] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/4b9f404daed84198a868845d13ba7c59
I20260929 15:52:05.088878 24314 ts_tablet_manager.cc:1939] T 4b9f404daed84198a868845d13ba7c59 P af29a2e9090c4a5fb36fe839cdd4646b: Deleting consensus metadata
I20260929 15:52:05.089287 24154 tablet_service.cc:1467] Processing CreateTablet for tablet bcb0094c3d574553913cf52e0d2d1e9a (DEFAULT_TABLE table=test-table [id=b9f5cfbad8c44de78cb988f3d3f14d60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:52:05.089569 24010 tablet_service.cc:1467] Processing CreateTablet for tablet bcb0094c3d574553913cf52e0d2d1e9a (DEFAULT_TABLE table=test-table [id=b9f5cfbad8c44de78cb988f3d3f14d60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:52:05.089745 24154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcb0094c3d574553913cf52e0d2d1e9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:05.090166 23870 tablet_service.cc:1467] Processing CreateTablet for tablet bcb0094c3d574553913cf52e0d2d1e9a (DEFAULT_TABLE table=test-table [id=b9f5cfbad8c44de78cb988f3d3f14d60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:52:05.090590 23870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcb0094c3d574553913cf52e0d2d1e9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:05.093633 24315 ts_tablet_manager.cc:1918] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:52:05.093789 24315 log.cc:1196] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/4b9f404daed84198a868845d13ba7c59
I20260929 15:52:05.094259 24315 ts_tablet_manager.cc:1939] T 4b9f404daed84198a868845d13ba7c59 P 563a2b2e105b4346b04102d7f35be39c: Deleting consensus metadata
I20260929 15:52:05.095458 23719 catalog_manager.cc:4841] TS 563a2b2e105b4346b04102d7f35be39c (127.15.196.195:42727): tablet 4b9f404daed84198a868845d13ba7c59 (table test-table [id=f94160e05276421c8a2161cb5f0256b0]) successfully deleted
I20260929 15:52:05.096807 24329 tablet_bootstrap.cc:492] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76: Bootstrap starting.
I20260929 15:52:05.097453 24010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcb0094c3d574553913cf52e0d2d1e9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:05.098301 24330 tablet_bootstrap.cc:492] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c: Bootstrap starting.
I20260929 15:52:05.098471 24329 tablet_bootstrap.cc:654] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:05.099071 24313 ts_tablet_manager.cc:1918] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:52:05.099211 24313 log.cc:1196] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/4b9f404daed84198a868845d13ba7c59
I20260929 15:52:05.099799 24313 ts_tablet_manager.cc:1939] T 4b9f404daed84198a868845d13ba7c59 P d72220c68c774184b366c2dacdca8f76: Deleting consensus metadata
I20260929 15:52:05.099906 24330 tablet_bootstrap.cc:654] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:05.101266 23719 catalog_manager.cc:4841] TS d72220c68c774184b366c2dacdca8f76 (127.15.196.193:33007): tablet 4b9f404daed84198a868845d13ba7c59 (table test-table [id=f94160e05276421c8a2161cb5f0256b0]) successfully deleted
W20260929 15:52:05.101897 23719 catalog_manager.cc:4808] TS d72220c68c774184b366c2dacdca8f76 (127.15.196.193:33007): delete failed for tablet 4b9f404daed84198a868845d13ba7c59 because the tablet was not found. No further retry: Not found: Tablet not found: 4b9f404daed84198a868845d13ba7c59
I20260929 15:52:05.104310 24328 tablet_bootstrap.cc:492] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b: Bootstrap starting.
I20260929 15:52:05.105891 24328 tablet_bootstrap.cc:654] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:05.105844 23720 catalog_manager.cc:4841] TS af29a2e9090c4a5fb36fe839cdd4646b (127.15.196.194:38361): tablet 4b9f404daed84198a868845d13ba7c59 (table test-table [id=f94160e05276421c8a2161cb5f0256b0]) successfully deleted
I20260929 15:52:05.111897 24330 tablet_bootstrap.cc:492] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c: No bootstrap required, opened a new log
I20260929 15:52:05.112084 24330 ts_tablet_manager.cc:1397] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.007s
I20260929 15:52:05.112877 24330 raft_consensus.cc:348] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:05.113142 24330 raft_consensus.cc:374] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:05.113233 24330 raft_consensus.cc:729] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 563a2b2e105b4346b04102d7f35be39c, State: Initialized, Role: FOLLOWER
I20260929 15:52:05.113487 24330 consensus_queue.cc:260] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:05.114655 24330 ts_tablet_manager.cc:1428] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 15:52:05.118623 24328 tablet_bootstrap.cc:492] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b: No bootstrap required, opened a new log
I20260929 15:52:05.118639 24329 tablet_bootstrap.cc:492] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76: No bootstrap required, opened a new log
I20260929 15:52:05.118808 24329 ts_tablet_manager.cc:1397] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76: Time spent bootstrapping tablet: real 0.022s user 0.012s sys 0.000s
I20260929 15:52:05.118808 24328 ts_tablet_manager.cc:1397] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.011s
I20260929 15:52:05.119626 24329 raft_consensus.cc:348] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:05.119623 24328 raft_consensus.cc:348] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:05.119887 24329 raft_consensus.cc:374] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:05.119887 24328 raft_consensus.cc:374] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:05.119967 24329 raft_consensus.cc:729] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72220c68c774184b366c2dacdca8f76, State: Initialized, Role: FOLLOWER
I20260929 15:52:05.119967 24328 raft_consensus.cc:729] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af29a2e9090c4a5fb36fe839cdd4646b, State: Initialized, Role: FOLLOWER
I20260929 15:52:05.120215 24329 consensus_queue.cc:260] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:05.120239 24328 consensus_queue.cc:260] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:05.121270 24328 ts_tablet_manager.cc:1428] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260929 15:52:05.121357 24329 ts_tablet_manager.cc:1428] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 15:52:05.188241 24267 raft_consensus.cc:482] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:52:05.188486 24267 raft_consensus.cc:504] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:05.189122 24267 leader_election.cc:288] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d72220c68c774184b366c2dacdca8f76 (127.15.196.193:33007), 563a2b2e105b4346b04102d7f35be39c (127.15.196.195:42727)
I20260929 15:52:05.189810 23894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb0094c3d574553913cf52e0d2d1e9a" candidate_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72220c68c774184b366c2dacdca8f76" is_pre_election: true
I20260929 15:52:05.189867 24175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb0094c3d574553913cf52e0d2d1e9a" candidate_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "563a2b2e105b4346b04102d7f35be39c" is_pre_election: true
I20260929 15:52:05.190110 23894 raft_consensus.cc:2454] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af29a2e9090c4a5fb36fe839cdd4646b in term 0.
I20260929 15:52:05.190186 24175 raft_consensus.cc:2454] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af29a2e9090c4a5fb36fe839cdd4646b in term 0.
I20260929 15:52:05.190626 23962 leader_election.cc:302] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af29a2e9090c4a5fb36fe839cdd4646b, d72220c68c774184b366c2dacdca8f76; no voters:
I20260929 15:52:05.190934 24267 raft_consensus.cc:2781] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:52:05.191051 24267 raft_consensus.cc:482] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:52:05.191179 24267 raft_consensus.cc:3037] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:05.193251 24267 raft_consensus.cc:504] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:05.193758 24267 leader_election.cc:288] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [CANDIDATE]: Term 1 election: Requested vote from peers d72220c68c774184b366c2dacdca8f76 (127.15.196.193:33007), 563a2b2e105b4346b04102d7f35be39c (127.15.196.195:42727)
I20260929 15:52:05.194224 23894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb0094c3d574553913cf52e0d2d1e9a" candidate_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d72220c68c774184b366c2dacdca8f76"
I20260929 15:52:05.194283 24175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb0094c3d574553913cf52e0d2d1e9a" candidate_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "563a2b2e105b4346b04102d7f35be39c"
I20260929 15:52:05.194428 23894 raft_consensus.cc:3037] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:05.194497 24175 raft_consensus.cc:3037] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:05.196482 23894 raft_consensus.cc:2454] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af29a2e9090c4a5fb36fe839cdd4646b in term 1.
I20260929 15:52:05.196501 24175 raft_consensus.cc:2454] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af29a2e9090c4a5fb36fe839cdd4646b in term 1.
I20260929 15:52:05.196930 23962 leader_election.cc:302] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af29a2e9090c4a5fb36fe839cdd4646b, d72220c68c774184b366c2dacdca8f76; no voters:
I20260929 15:52:05.197201 24267 raft_consensus.cc:2781] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:52:05.197386 24267 raft_consensus.cc:686] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [term 1 LEADER]: Becoming Leader. State: Replica: af29a2e9090c4a5fb36fe839cdd4646b, State: Running, Role: LEADER
I20260929 15:52:05.197676 24267 consensus_queue.cc:237] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } }
I20260929 15:52:05.200356 23732 catalog_manager.cc:5495] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b reported cstate change: term changed from 0 to 1, leader changed from <none> to af29a2e9090c4a5fb36fe839cdd4646b (127.15.196.194). New cstate: current_term: 1 leader_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af29a2e9090c4a5fb36fe839cdd4646b" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 } health_report { overall_health: UNKNOWN } } }
I20260929 15:52:05.244951 24175 raft_consensus.cc:1261] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Refusing update from remote peer af29a2e9090c4a5fb36fe839cdd4646b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:52:05.245604 23894 raft_consensus.cc:1261] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Refusing update from remote peer af29a2e9090c4a5fb36fe839cdd4646b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:52:05.245608 24255 consensus_queue.cc:1035] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [LEADER]: Connected to new peer: Peer: permanent_uuid: "563a2b2e105b4346b04102d7f35be39c" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 42727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:52:05.246100 24255 consensus_queue.cc:1035] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72220c68c774184b366c2dacdca8f76" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:52:05.317783 23733 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:52:05.318027 23733 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:52:05.320922 23733 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5aa785f705747b7abb64b27a8f627c3: Sending DeleteTablet for 3 replicas of tablet bcb0094c3d574553913cf52e0d2d1e9a
I20260929 15:52:05.321688 24010 tablet_service.cc:1514] Processing DeleteTablet for tablet bcb0094c3d574553913cf52e0d2d1e9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:52:05 UTC) from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:55758
I20260929 15:52:05.321828 23870 tablet_service.cc:1514] Processing DeleteTablet for tablet bcb0094c3d574553913cf52e0d2d1e9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:52:05 UTC) from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:37970
I20260929 15:52:05.321780 24154 tablet_service.cc:1514] Processing DeleteTablet for tablet bcb0094c3d574553913cf52e0d2d1e9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:52:05 UTC) from {username='oryx', principal='oryx/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:56452
I20260929 15:52:05.322047 24314 tablet_replica.cc:330] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b: stopping tablet replica
I20260929 15:52:05.322151 24313 tablet_replica.cc:330] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76: stopping tablet replica
I20260929 15:52:05.322175 24315 tablet_replica.cc:330] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c: stopping tablet replica
I20260929 15:52:05.322234 24314 raft_consensus.cc:2229] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:52:05.322312 24313 raft_consensus.cc:2229] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:52:05.322368 24315 raft_consensus.cc:2229] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:52:05.322500 24313 raft_consensus.cc:2258] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:52:05.322573 24315 raft_consensus.cc:2258] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:52:05.322580 24314 raft_consensus.cc:2258] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:52:05.323534 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 23805
I20260929 15:52:05.323619 24313 ts_tablet_manager.cc:1905] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:52:05.323732 24314 ts_tablet_manager.cc:1905] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:52:05.323843 24315 ts_tablet_manager.cc:1905] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:52:05.330255 24313 ts_tablet_manager.cc:1918] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 15:52:05.330401 24313 log.cc:1196] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/bcb0094c3d574553913cf52e0d2d1e9a
I20260929 15:52:05.330681 24314 ts_tablet_manager.cc:1918] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 15:52:05.330847 24314 log.cc:1196] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/bcb0094c3d574553913cf52e0d2d1e9a
I20260929 15:52:05.330859 24313 ts_tablet_manager.cc:1939] T bcb0094c3d574553913cf52e0d2d1e9a P d72220c68c774184b366c2dacdca8f76: Deleting consensus metadata
I20260929 15:52:05.331327 24314 ts_tablet_manager.cc:1939] T bcb0094c3d574553913cf52e0d2d1e9a P af29a2e9090c4a5fb36fe839cdd4646b: Deleting consensus metadata
I20260929 15:52:05.331643 24315 ts_tablet_manager.cc:1918] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 15:52:05.331806 24315 log.cc:1196] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/bcb0094c3d574553913cf52e0d2d1e9a
I20260929 15:52:05.331982 23719 catalog_manager.cc:4841] TS d72220c68c774184b366c2dacdca8f76 (127.15.196.193:33007): tablet bcb0094c3d574553913cf52e0d2d1e9a (table test-table [id=b9f5cfbad8c44de78cb988f3d3f14d60]) successfully deleted
I20260929 15:52:05.332270 24315 ts_tablet_manager.cc:1939] T bcb0094c3d574553913cf52e0d2d1e9a P 563a2b2e105b4346b04102d7f35be39c: Deleting consensus metadata
I20260929 15:52:05.332610 23720 catalog_manager.cc:4841] TS af29a2e9090c4a5fb36fe839cdd4646b (127.15.196.194:38361): tablet bcb0094c3d574553913cf52e0d2d1e9a (table test-table [id=b9f5cfbad8c44de78cb988f3d3f14d60]) successfully deleted
I20260929 15:52:05.333591 23719 catalog_manager.cc:4841] TS 563a2b2e105b4346b04102d7f35be39c (127.15.196.195:42727): tablet bcb0094c3d574553913cf52e0d2d1e9a (table test-table [id=b9f5cfbad8c44de78cb988f3d3f14d60]) successfully deleted
Sep 29 15:52:05 dist-test-slave-c51v krb5kdc[23689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.196.193@KRBTEST.COM
I20260929 15:52:05.347575 24350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:05.325144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37974 (local address 127.15.196.193:33007)
0929 15:52:05.333630 (+ 8486us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:05.333636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:05.333653 (+ 17us) server_negotiation.cc:407] Connection header received
0929 15:52:05.333708 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:05.333713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:05.333798 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:05.333881 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:05.335055 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:05.335796 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:05.336995 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:05.337419 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:05.337510 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:05.341976 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:05.341991 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:05.342008 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:05.342061 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:05.344790 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:05.345503 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:05.345511 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:05.345514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:05.345644 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:05.346005 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:05.346011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:05.346014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:05.346366 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:05.346663 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:05.346963 (+ 300us) server_negotiation.cc:299] Negotiation successful
0929 15:52:05.347188 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8364,"thread_start_us":8267,"threads_started":1}
I20260929 15:52:05.348251 23936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:05.445427 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 23805
I20260929 15:52:05.463790 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 23945
Sep 29 15:52:05 dist-test-slave-c51v krb5kdc[23689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.196.194@KRBTEST.COM
I20260929 15:52:05.477906 24241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:05.464580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55788 (local address 127.15.196.194:38361)
0929 15:52:05.464753 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:05.464757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:05.464774 (+ 17us) server_negotiation.cc:407] Connection header received
0929 15:52:05.464860 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:05.464864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:05.464924 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:05.465009 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:05.466191 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:05.466900 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:05.468128 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:05.468499 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:05.468597 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:05.472762 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:05.472775 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:05.472790 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:05.472822 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:05.475752 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:05.476335 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:05.476341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:05.476343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:05.476459 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:05.476756 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:05.476759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:05.476761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:05.477092 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:05.477317 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:05.477499 (+ 182us) server_negotiation.cc:299] Negotiation successful
0929 15:52:05.477674 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 15:52:05.478576 24075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:05.605077 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 23945
I20260929 15:52:05.627957 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 24084
Sep 29 15:52:05 dist-test-slave-c51v krb5kdc[23689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.196.195@KRBTEST.COM
I20260929 15:52:05.642338 24353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:05.628668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56468 (local address 127.15.196.195:42727)
0929 15:52:05.629147 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:05.629152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:05.629169 (+ 17us) server_negotiation.cc:407] Connection header received
0929 15:52:05.629224 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:05.629227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:05.629284 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:05.629374 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:05.630575 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:05.631267 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:05.632591 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:05.632984 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:05.633094 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:05.637251 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:05.637266 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:05.637282 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:05.637314 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:05.639996 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:05.640598 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:05.640605 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:05.640607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:05.640728 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:05.641027 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:05.641030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:05.641032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:05.641361 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:05.641580 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:05.641793 (+ 213us) server_negotiation.cc:299] Negotiation successful
0929 15:52:05.641959 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":269,"threads_started":1}
I20260929 15:52:05.642961 24215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:05.765938 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 24084
I20260929 15:52:05.787465 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 23702
I20260929 15:52:05.788698 23774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:05.905649 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 23702
2026-09-29T15:52:05Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5624 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:52:06 dist-test-slave-c51v krb5kdc[24358](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:52:06 dist-test-slave-c51v krb5kdc[24358](info): set up 2 sockets
Sep 29 15:52:06 dist-test-slave-c51v krb5kdc[24358](info): commencing operation
krb5kdc: starting...
W20260929 15:52:07.006326 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:52:07 dist-test-slave-c51v krb5kdc[24358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:52:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:52:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:52:07.231491 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:33475
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:36347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:33475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:52:07.549712 24371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:07.550009 24371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:07.550232 24371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:07.557421 24371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:52:07.557559 24371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:07.557667 24371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:07.557765 24371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:52:07.557857 24371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:52:07.570499 24371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:36347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:33475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:33475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:07.571920 24371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:07.573256 24371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:07.584896 24376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:07.584856 24379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:07.584923 24377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:07.586215 24371 server_base.cc:1034] running on GCE node
I20260929 15:52:07.587006 24371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:07.588546 24371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:07.589785 24371 hybrid_clock.cc:648] HybridClock initialized: now 1790697127589756 us; error 36 us; skew 500 ppm
I20260929 15:52:07.590144 24371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-29T15:52:07Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1751 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:52:07 dist-test-slave-c51v krb5kdc[24385](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:52:07 dist-test-slave-c51v krb5kdc[24385](info): set up 2 sockets
Sep 29 15:52:07 dist-test-slave-c51v krb5kdc[24385](info): commencing operation
krb5kdc: starting...
W20260929 15:52:08.770340 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:52:08 dist-test-slave-c51v krb5kdc[24385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:52:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:52:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:52:08.995718 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:36891
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:36891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:52:09.320817 24398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:09.321126 24398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:09.321308 24398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:09.328632 24398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:52:09.328744 24398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:09.328855 24398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:09.328953 24398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:52:09.329046 24398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:52:09.339007 24398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:36891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:36891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:09.340420 24398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:09.341753 24398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:09.353636 24403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:09.353615 24404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:09.353695 24406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:09.354843 24398 server_base.cc:1034] running on GCE node
I20260929 15:52:09.355726 24398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:09.357128 24398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:09.358364 24398 hybrid_clock.cc:648] HybridClock initialized: now 1790697129358291 us; error 52 us; skew 500 ppm
I20260929 15:52:09.358723 24398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:52:09 dist-test-slave-c51v krb5kdc[24385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697129, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:52:09.365073 24398 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:52:09.366657 24398 webserver.cc:460] Webserver started at http://127.15.196.254:43483/ using document root <none> and password file <none>
I20260929 15:52:09.367445 24398 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:09.367564 24398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:09.367844 24398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:09.371075 24398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "8f6ca6e85674452c9aaf391493fa4cac"
format_stamp: "Formatted at 2026-09-29 15:52:09 on dist-test-slave-c51v"
server_key: "19e9e426f3a3353d67d5c7687a571856"
server_key_iv: "2acf264a47e808c9ec5d233da4cd236a"
server_key_version: "encryptionkey@0"
I20260929 15:52:09.371896 24398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "8f6ca6e85674452c9aaf391493fa4cac"
format_stamp: "Formatted at 2026-09-29 15:52:09 on dist-test-slave-c51v"
server_key: "19e9e426f3a3353d67d5c7687a571856"
server_key_iv: "2acf264a47e808c9ec5d233da4cd236a"
server_key_version: "encryptionkey@0"
I20260929 15:52:09.377426 24398 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:52:09.381116 24413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:09.382480 24398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 15:52:09.382687 24398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "8f6ca6e85674452c9aaf391493fa4cac"
format_stamp: "Formatted at 2026-09-29 15:52:09 on dist-test-slave-c51v"
server_key: "19e9e426f3a3353d67d5c7687a571856"
server_key_iv: "2acf264a47e808c9ec5d233da4cd236a"
server_key_version: "encryptionkey@0"
I20260929 15:52:09.382898 24398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:09.406919 24398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:09.411700 24398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:09.412015 24398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:09.431905 24398 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:36891
I20260929 15:52:09.431926 24464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:36891 every 8 connection(s)
I20260929 15:52:09.433391 24398 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:52:09.437192 24465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:09.437326 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 24398
I20260929 15:52:09.437572 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:52:09.438300 16147 external_mini_cluster.cc:1428] Setting key 33c3ce0cd9891f174dffed42507d327c
I20260929 15:52:09.451983 24465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac: Bootstrap starting.
Sep 29 15:52:09 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:52:09.455206 24465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:09.456324 24465 log.cc:824] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac: Log is configured to *not* fsync() on all Append() calls
I20260929 15:52:09.459062 24465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac: No bootstrap required, opened a new log
I20260929 15:52:09.461400 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:09.440414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57026 (local address 127.15.196.254:36891)
0929 15:52:09.441252 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:09.441264 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:09.441309 (+ 45us) server_negotiation.cc:407] Connection header received
0929 15:52:09.443081 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:09.443108 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:09.443599 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:09.443958 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:09.445829 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:09.447072 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:09.448932 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:09.449618 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:09.449748 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:09.454208 (+ 4460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:09.454215 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:09.454230 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:09.454265 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:09.457418 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:09.458136 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:09.458143 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:09.458149 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:09.458301 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:09.458685 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:09.458690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:09.458692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:09.459213 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:09.459602 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:09.460010 (+ 408us) server_negotiation.cc:299] Negotiation successful
0929 15:52:09.460413 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":399,"threads_started":1}
I20260929 15:52:09.467792 24465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6ca6e85674452c9aaf391493fa4cac" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36891 } }
I20260929 15:52:09.468173 24465 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:09.468277 24465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f6ca6e85674452c9aaf391493fa4cac, State: Initialized, Role: FOLLOWER
I20260929 15:52:09.468865 24465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6ca6e85674452c9aaf391493fa4cac" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36891 } }
I20260929 15:52:09.469137 24465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:52:09.469295 24465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:52:09.469497 24465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:09.471904 24465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6ca6e85674452c9aaf391493fa4cac" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36891 } }
I20260929 15:52:09.472381 24465 leader_election.cc:302] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f6ca6e85674452c9aaf391493fa4cac; no voters:
I20260929 15:52:09.472922 24465 leader_election.cc:288] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:52:09.473071 24470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:52:09.473752 24470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 1 LEADER]: Becoming Leader. State: Replica: 8f6ca6e85674452c9aaf391493fa4cac, State: Running, Role: LEADER
I20260929 15:52:09.474262 24470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6ca6e85674452c9aaf391493fa4cac" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36891 } }
I20260929 15:52:09.475069 24465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:52:09.477346 24471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f6ca6e85674452c9aaf391493fa4cac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6ca6e85674452c9aaf391493fa4cac" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36891 } } }
I20260929 15:52:09.477769 24471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [sys.catalog]: This master's current role is: LEADER
I20260929 15:52:09.478201 24472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f6ca6e85674452c9aaf391493fa4cac. Latest consensus state: current_term: 1 leader_uuid: "8f6ca6e85674452c9aaf391493fa4cac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6ca6e85674452c9aaf391493fa4cac" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36891 } } }
I20260929 15:52:09.478607 24472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [sys.catalog]: This master's current role is: LEADER
I20260929 15:52:09.479889 24478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:52:09.485107 24478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:52:09.494642 24478 catalog_manager.cc:1347] Generated new cluster ID: 8f6f34f934bb476aabfde6d86473ae61
I20260929 15:52:09.494791 24478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:52:09.504417 24478 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:52:09.505765 24478 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:52:09.520680 24478 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac: Generated new TSK 0
I20260929 15:52:09.521905 24478 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:52:09.587769 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:36891
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:52:09.916316 24491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:09.916694 24491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:09.916864 24491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:09.924347 24491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:09.924516 24491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:09.924712 24491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:52:09.934540 24491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:36891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:09.935968 24491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:09.937294 24491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:09.950481 24499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:09.950459 24497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:09.950795 24496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:09.952042 24491 server_base.cc:1034] running on GCE node
I20260929 15:52:09.952674 24491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:09.953539 24491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:09.954746 24491 hybrid_clock.cc:648] HybridClock initialized: now 1790697129954707 us; error 36 us; skew 500 ppm
I20260929 15:52:09.955107 24491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:52:09 dist-test-slave-c51v krb5kdc[24385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697129, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:52:09.958187 24491 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:52:09.959605 24491 webserver.cc:460] Webserver started at http://127.15.196.193:36525/ using document root <none> and password file <none>
I20260929 15:52:09.960268 24491 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:09.960423 24491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:09.960736 24491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:09.963697 24491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "19ba08e78b27499b84acdd11b73efb1a"
format_stamp: "Formatted at 2026-09-29 15:52:09 on dist-test-slave-c51v"
server_key: "fae3e1f0538bfe0e6a8520048b66e17d"
server_key_iv: "61d066d9da7e910cce678fb166b23c93"
server_key_version: "encryptionkey@0"
I20260929 15:52:09.964447 24491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "19ba08e78b27499b84acdd11b73efb1a"
format_stamp: "Formatted at 2026-09-29 15:52:09 on dist-test-slave-c51v"
server_key: "fae3e1f0538bfe0e6a8520048b66e17d"
server_key_iv: "61d066d9da7e910cce678fb166b23c93"
server_key_version: "encryptionkey@0"
I20260929 15:52:09.969481 24491 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260929 15:52:09.972860 24506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:09.974078 24491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 15:52:09.974251 24491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "19ba08e78b27499b84acdd11b73efb1a"
format_stamp: "Formatted at 2026-09-29 15:52:09 on dist-test-slave-c51v"
server_key: "fae3e1f0538bfe0e6a8520048b66e17d"
server_key_iv: "61d066d9da7e910cce678fb166b23c93"
server_key_version: "encryptionkey@0"
I20260929 15:52:09.974447 24491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:09.990928 24491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:09.994799 24491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:09.995095 24491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:09.996073 24491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:52:09.997388 24491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:09.997493 24491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:09.997658 24491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:09.997731 24491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:10.030508 24491 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:45543
I20260929 15:52:10.030557 24618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:45543 every 8 connection(s)
I20260929 15:52:10.031939 24491 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:52:10.042121 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 24491
I20260929 15:52:10.042347 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:52:10.042979 16147 external_mini_cluster.cc:1428] Setting key d0c9cbda79a1d42440af0a2ea14ccb57
Sep 29 15:52:10 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697129, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:52:10.052923 24621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:10.034474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:43865 (local address 127.15.196.254:36891)
0929 15:52:10.034885 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:10.034890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:10.035982 (+ 1092us) server_negotiation.cc:407] Connection header received
0929 15:52:10.037760 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:10.037766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:10.037856 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:10.037970 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:10.039455 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:10.040190 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:10.041720 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:10.042108 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:10.042247 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:10.046057 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:10.046066 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:10.046074 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:10.046117 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:10.049655 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:10.050389 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:10.050397 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:10.050404 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:10.050561 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:10.050966 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:10.050974 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:10.050979 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:10.051389 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:10.051534 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:10.052235 (+ 701us) server_negotiation.cc:299] Negotiation successful
0929 15:52:10.052492 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":212,"threads_started":1}
I20260929 15:52:10.053920 24622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:10.035075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:43865)
0929 15:52:10.035806 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:10.035844 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:10.037491 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:10.038210 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:10.038219 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:10.038710 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:10.039246 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:10.039262 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:10.040355 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:10.040360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:10.041572 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:10.041585 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:10.042336 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:10.042345 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:10.042599 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:10.043333 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:10.043373 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:10.045792 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:10.049808 (+ 4016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:10.049815 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:10.049829 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:10.050253 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:10.050667 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:10.050670 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:10.050672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:10.050850 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:10.051517 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:10.051523 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:10.051865 (+ 342us) client_negotiation.cc:769] Sending connection context
0929 15:52:10.052092 (+ 227us) client_negotiation.cc:240] Negotiation successful
0929 15:52:10.052471 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":327,"threads_started":1}
I20260929 15:52:10.055490 24619 heartbeater.cc:347] Connected to a master server at 127.15.196.254:36891
I20260929 15:52:10.055799 24619 heartbeater.cc:464] Registering TS with master...
I20260929 15:52:10.056511 24619 heartbeater.cc:511] Master 127.15.196.254:36891 requested a full tablet report, sending...
I20260929 15:52:10.058560 24430 ts_manager.cc:194] Registered new tserver with Master: 19ba08e78b27499b84acdd11b73efb1a (127.15.196.193:45543)
I20260929 15:52:10.060047 24430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:43865
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:52:10.094344 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:36891
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:52:10.428000 24625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:10.428349 24625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:10.428540 24625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:10.436188 24625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:10.436357 24625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:10.436543 24625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:52:10.446643 24625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:36891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:10.448048 24625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:10.449405 24625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:10.463346 24631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:10.463332 24630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:10.463372 24633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:10.464203 24625 server_base.cc:1034] running on GCE node
I20260929 15:52:10.464738 24625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:10.465590 24625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:10.466780 24625 hybrid_clock.cc:648] HybridClock initialized: now 1790697130466731 us; error 42 us; skew 500 ppm
I20260929 15:52:10.467041 24625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:52:10 dist-test-slave-c51v krb5kdc[24385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697130, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:52:10.470403 24625 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:52:10.471925 24625 webserver.cc:460] Webserver started at http://127.15.196.194:46069/ using document root <none> and password file <none>
I20260929 15:52:10.472617 24625 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:10.472740 24625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:10.473057 24625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:10.476176 24625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "533240e3d3bb4870873f254be9a52e17"
format_stamp: "Formatted at 2026-09-29 15:52:10 on dist-test-slave-c51v"
server_key: "9b5c3d1e175742b3d071f2cbd510a5bc"
server_key_iv: "4c0d87cde61fdf2b9cc9ad44c7e27352"
server_key_version: "encryptionkey@0"
I20260929 15:52:10.476886 24625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "533240e3d3bb4870873f254be9a52e17"
format_stamp: "Formatted at 2026-09-29 15:52:10 on dist-test-slave-c51v"
server_key: "9b5c3d1e175742b3d071f2cbd510a5bc"
server_key_iv: "4c0d87cde61fdf2b9cc9ad44c7e27352"
server_key_version: "encryptionkey@0"
I20260929 15:52:10.482172 24625 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 15:52:10.485491 24640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:10.486805 24625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 15:52:10.486979 24625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "533240e3d3bb4870873f254be9a52e17"
format_stamp: "Formatted at 2026-09-29 15:52:10 on dist-test-slave-c51v"
server_key: "9b5c3d1e175742b3d071f2cbd510a5bc"
server_key_iv: "4c0d87cde61fdf2b9cc9ad44c7e27352"
server_key_version: "encryptionkey@0"
I20260929 15:52:10.487180 24625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:10.504304 24625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:10.508757 24625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:10.509049 24625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:10.510010 24625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:52:10.511409 24625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:10.511505 24625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:10.511660 24625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:10.511732 24625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:10.544217 24625 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:44389
I20260929 15:52:10.544239 24752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:44389 every 8 connection(s)
I20260929 15:52:10.545584 24625 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:52:10.554919 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 24625
I20260929 15:52:10.555114 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:52:10.555627 16147 external_mini_cluster.cc:1428] Setting key b17617343d7d6899fa5bd8e1ff3a8f96
Sep 29 15:52:10 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697130, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:52:10.565224 24621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:10.548047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:45035 (local address 127.15.196.254:36891)
0929 15:52:10.548231 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:10.548235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:10.549408 (+ 1173us) server_negotiation.cc:407] Connection header received
0929 15:52:10.551159 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:10.551164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:10.551236 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:10.551457 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:10.552860 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:10.553550 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:10.555264 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:10.555778 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:10.555921 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:10.559515 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:10.559520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:10.559524 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:10.559557 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:10.562150 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:10.562889 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:10.562900 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:10.562902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:10.563028 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:10.563510 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:10.563516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:10.563520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:10.563788 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:10.563943 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:10.564653 (+ 710us) server_negotiation.cc:299] Negotiation successful
0929 15:52:10.564883 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 15:52:10.566355 24755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:10.548563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:45035)
0929 15:52:10.549244 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:10.549287 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:10.550883 (+ 1596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:10.551646 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:10.551654 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:10.552138 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:10.552698 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:10.552715 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:10.553680 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:10.553683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:10.555119 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:10.555131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:10.556001 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:10.556008 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:10.556241 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:10.556994 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:10.557016 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:10.559251 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:10.562295 (+ 3044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:10.562305 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:10.562324 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:10.562759 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:10.563138 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:10.563145 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:10.563149 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:10.563404 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:10.563914 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:10.563923 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:10.564289 (+ 366us) client_negotiation.cc:769] Sending connection context
0929 15:52:10.564578 (+ 289us) client_negotiation.cc:240] Negotiation successful
0929 15:52:10.565117 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":267,"threads_started":1}
I20260929 15:52:10.568041 24753 heartbeater.cc:347] Connected to a master server at 127.15.196.254:36891
I20260929 15:52:10.568380 24753 heartbeater.cc:464] Registering TS with master...
I20260929 15:52:10.569124 24753 heartbeater.cc:511] Master 127.15.196.254:36891 requested a full tablet report, sending...
I20260929 15:52:10.570626 24430 ts_manager.cc:194] Registered new tserver with Master: 533240e3d3bb4870873f254be9a52e17 (127.15.196.194:44389)
I20260929 15:52:10.571522 24430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:45035
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:52:10.606835 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:36891
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:52:10.933668 24758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:10.933987 24758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:10.934084 24758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:10.943648 24758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:10.943806 24758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:10.944087 24758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:52:10.955621 24758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:36891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:10.956868 24758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:10.958218 24758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:10.971061 24763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:10.971133 24764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:10.971169 24766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:10.972458 24758 server_base.cc:1034] running on GCE node
I20260929 15:52:10.973122 24758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:10.974087 24758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:10.975365 24758 hybrid_clock.cc:648] HybridClock initialized: now 1790697130975250 us; error 50 us; skew 500 ppm
I20260929 15:52:10.975723 24758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:52:10 dist-test-slave-c51v krb5kdc[24385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697130, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:52:10.979024 24758 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:52:10.980646 24758 webserver.cc:460] Webserver started at http://127.15.196.195:37047/ using document root <none> and password file <none>
I20260929 15:52:10.981360 24758 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:10.981500 24758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:10.981824 24758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:10.984792 24758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "f206f25ee4e641e1ba708f3424052dea"
format_stamp: "Formatted at 2026-09-29 15:52:10 on dist-test-slave-c51v"
server_key: "db3b6cdafedc1307f492d6b924e78afd"
server_key_iv: "b68444f162a7f92ebf75e5e765766bd9"
server_key_version: "encryptionkey@0"
I20260929 15:52:10.985582 24758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "f206f25ee4e641e1ba708f3424052dea"
format_stamp: "Formatted at 2026-09-29 15:52:10 on dist-test-slave-c51v"
server_key: "db3b6cdafedc1307f492d6b924e78afd"
server_key_iv: "b68444f162a7f92ebf75e5e765766bd9"
server_key_version: "encryptionkey@0"
I20260929 15:52:10.990957 24758 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 15:52:10.994424 24773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:10.995878 24758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:52:10.996073 24758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "f206f25ee4e641e1ba708f3424052dea"
format_stamp: "Formatted at 2026-09-29 15:52:10 on dist-test-slave-c51v"
server_key: "db3b6cdafedc1307f492d6b924e78afd"
server_key_iv: "b68444f162a7f92ebf75e5e765766bd9"
server_key_version: "encryptionkey@0"
I20260929 15:52:10.996281 24758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:11.032858 24758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:11.037336 24758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:11.037644 24758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:11.038669 24758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:52:11.040078 24758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:11.040181 24758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:11.040339 24758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:11.040413 24758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:11.063089 24619 heartbeater.cc:503] Master 127.15.196.254:36891 was elected leader, sending a full tablet report...
I20260929 15:52:11.073071 24758 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:42001
I20260929 15:52:11.073118 24885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:42001 every 8 connection(s)
I20260929 15:52:11.074383 24758 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:52:11.079478 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 24758
I20260929 15:52:11.079689 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:52:11.080295 16147 external_mini_cluster.cc:1428] Setting key f11146f0d4f6392ddeb8fc930ecda0d7
Sep 29 15:52:11 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697130, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:52:11.095736 24888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:11.077067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:38079 (local address 127.15.196.254:36891)
0929 15:52:11.077546 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:11.077553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:11.078901 (+ 1348us) server_negotiation.cc:407] Connection header received
0929 15:52:11.081687 (+ 2786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:11.081693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:11.081788 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:11.081907 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:11.083657 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:11.084571 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:11.086060 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:11.086650 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:11.086755 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:11.090047 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:11.090052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:11.090055 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:11.090086 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:11.092870 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:11.093525 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:11.093529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:11.093531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:11.093651 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:11.094017 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:11.094021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:11.094024 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:11.094347 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:11.094489 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:11.095139 (+ 650us) server_negotiation.cc:299] Negotiation successful
0929 15:52:11.095379 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":245,"threads_started":1}
I20260929 15:52:11.096728 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:11.077870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:38079)
0929 15:52:11.078726 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:11.078770 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:11.081360 (+ 2590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:11.082148 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:11.082158 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:11.082785 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:11.083457 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:11.083484 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:11.084695 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:11.084699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:11.085924 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:11.085935 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:11.086839 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:11.086844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:11.087036 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:11.087728 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:11.087754 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:11.089859 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:11.092986 (+ 3127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:11.092992 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:11.093004 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:11.093420 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:11.093738 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:11.093741 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:11.093743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:11.093903 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:11.094479 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:11.094485 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:11.094785 (+ 300us) client_negotiation.cc:769] Sending connection context
0929 15:52:11.094999 (+ 214us) client_negotiation.cc:240] Negotiation successful
0929 15:52:11.095526 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":301,"threads_started":1}
I20260929 15:52:11.098037 24886 heartbeater.cc:347] Connected to a master server at 127.15.196.254:36891
I20260929 15:52:11.098301 24886 heartbeater.cc:464] Registering TS with master...
I20260929 15:52:11.098950 24886 heartbeater.cc:511] Master 127.15.196.254:36891 requested a full tablet report, sending...
I20260929 15:52:11.100374 24430 ts_manager.cc:194] Registered new tserver with Master: f206f25ee4e641e1ba708f3424052dea (127.15.196.195:42001)
I20260929 15:52:11.101199 24430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:38079
I20260929 15:52:11.108274 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.193.keytab.
Entry for principal oryx/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.193.keytab.
WARNING: no policy specified for oryx/127.15.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.194.keytab.
Entry for principal oryx/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.194.keytab.
WARNING: no policy specified for oryx/127.15.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.195.keytab.
Entry for principal oryx/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.195.keytab.
I20260929 15:52:11.242441 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 24491
Sep 29 15:52:11 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:52:11.259117 24897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:11.243722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42908 (local address 127.15.196.193:45543)
0929 15:52:11.244550 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:11.244557 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:11.244583 (+ 26us) server_negotiation.cc:407] Connection header received
0929 15:52:11.244657 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:11.244664 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:11.244885 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:11.245017 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:11.246317 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:11.247368 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:11.248746 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:11.249289 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:11.249395 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:11.253530 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:11.253550 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:11.253566 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:11.253601 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:11.256449 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:11.257191 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:11.257197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:11.257200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:11.257327 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:11.257666 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:11.257669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:11.257671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:11.257997 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:11.258315 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:11.258478 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 15:52:11.258743 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":478,"threads_started":1}
I20260929 15:52:11.260706 24614 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:11.347216 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 24491
I20260929 15:52:11.363682 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 24625
Sep 29 15:52:11 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:52:11.379943 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:11.364611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47940 (local address 127.15.196.194:44389)
0929 15:52:11.365157 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:11.365164 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:11.365188 (+ 24us) server_negotiation.cc:407] Connection header received
0929 15:52:11.365253 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:11.365260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:11.365473 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:11.365650 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:11.366890 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:11.367988 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:11.369277 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:11.369832 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:11.369972 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:11.374120 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:11.374130 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:11.374144 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:11.374179 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:11.377002 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:11.377715 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:11.377723 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:11.377725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:11.377853 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:11.378219 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:11.378226 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:11.378229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:11.378559 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:11.378898 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:11.379241 (+ 343us) server_negotiation.cc:299] Negotiation successful
0929 15:52:11.379531 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":275,"threads_started":1}
I20260929 15:52:11.381220 24748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:11.463743 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 24625
I20260929 15:52:11.479952 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 24758
Sep 29 15:52:11 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:52:11.500226 24901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:11.480988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55680 (local address 127.15.196.195:42001)
0929 15:52:11.481605 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:11.481612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:11.481638 (+ 26us) server_negotiation.cc:407] Connection header received
0929 15:52:11.481698 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:11.481705 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:11.481906 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:11.482032 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:11.483280 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:11.484361 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:11.485555 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:11.486106 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:11.486198 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:11.494702 (+ 8504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:11.494714 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:11.494730 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:11.494771 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:11.497536 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:11.498222 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:11.498230 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:11.498232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:11.498350 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:11.498684 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:11.498686 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:11.498688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:11.499009 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:11.499231 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:11.499677 (+ 446us) server_negotiation.cc:299] Negotiation successful
0929 15:52:11.499938 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":320,"threads_started":1}
I20260929 15:52:11.501631 24881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:11.584862 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 24758
I20260929 15:52:11.600800 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 24398
I20260929 15:52:11.601979 24460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:11.705559 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 24398
I20260929 15:52:11.726511 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:36891
--webserver_interface=127.15.196.254
--webserver_port=43483
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:36891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:52:12.074496 24904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:12.074877 24904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:12.075057 24904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:12.082509 24904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:52:12.082669 24904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:12.082782 24904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:12.082886 24904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:52:12.082983 24904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:52:12.093076 24904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:36891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:36891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=43483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:12.094496 24904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:12.095865 24904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:12.108215 24912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:12.108171 24909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:12.108261 24910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:12.109110 24904 server_base.cc:1034] running on GCE node
I20260929 15:52:12.109875 24904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:12.111279 24904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:12.112674 24904 hybrid_clock.cc:648] HybridClock initialized: now 1790697132112593 us; error 53 us; skew 500 ppm
I20260929 15:52:12.113128 24904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:52:12 dist-test-slave-c51v krb5kdc[24385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697132, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:52:12.116475 24904 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:52:12.118148 24904 webserver.cc:460] Webserver started at http://127.15.196.254:43483/ using document root <none> and password file <none>
I20260929 15:52:12.118904 24904 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:12.119079 24904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:12.125011 24904 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 15:52:12.128149 24919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:12.129603 24904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:52:12.129823 24904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "8f6ca6e85674452c9aaf391493fa4cac"
format_stamp: "Formatted at 2026-09-29 15:52:09 on dist-test-slave-c51v"
server_key: "19e9e426f3a3353d67d5c7687a571856"
server_key_iv: "2acf264a47e808c9ec5d233da4cd236a"
server_key_version: "encryptionkey@0"
I20260929 15:52:12.130998 24904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:12.160079 24904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:12.164192 24904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:12.164459 24904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:12.183032 24904 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:36891
I20260929 15:52:12.183089 24970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:36891 every 8 connection(s)
I20260929 15:52:12.184574 24904 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:52:12.187817 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 24904
I20260929 15:52:12.188616 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:45543
--local_ip_for_outbound_sockets=127.15.196.193
--tserver_master_addrs=127.15.196.254:36891
--webserver_port=36525
--webserver_interface=127.15.196.193
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.196.193
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.193.keytab with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:52:12.191958 24971 sys_catalog.cc:263] Verifying existing consensus state
I20260929 15:52:12.195420 24971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac: Bootstrap starting.
I20260929 15:52:12.215189 24971 log.cc:824] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac: Log is configured to *not* fsync() on all Append() calls
I20260929 15:52:12.224709 24971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 15:52:12.225282 24971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac: Bootstrap complete.
I20260929 15:52:12.234629 24971 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6ca6e85674452c9aaf391493fa4cac" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36891 } }
I20260929 15:52:12.235756 24971 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f6ca6e85674452c9aaf391493fa4cac, State: Initialized, Role: FOLLOWER
I20260929 15:52:12.236467 24971 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6ca6e85674452c9aaf391493fa4cac" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36891 } }
I20260929 15:52:12.236719 24971 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:52:12.236850 24971 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:52:12.237051 24971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 1 FOLLOWER]: Advancing to term 2
I20260929 15:52:12.240942 24971 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6ca6e85674452c9aaf391493fa4cac" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36891 } }
I20260929 15:52:12.241443 24971 leader_election.cc:302] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f6ca6e85674452c9aaf391493fa4cac; no voters:
I20260929 15:52:12.242082 24971 leader_election.cc:288] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 15:52:12.242218 24975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 2 FOLLOWER]: Leader election won for term 2
I20260929 15:52:12.243577 24975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [term 2 LEADER]: Becoming Leader. State: Replica: 8f6ca6e85674452c9aaf391493fa4cac, State: Running, Role: LEADER
I20260929 15:52:12.244187 24975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6ca6e85674452c9aaf391493fa4cac" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36891 } }
I20260929 15:52:12.244375 24971 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:52:12.246804 24976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8f6ca6e85674452c9aaf391493fa4cac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6ca6e85674452c9aaf391493fa4cac" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36891 } } }
I20260929 15:52:12.247253 24976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [sys.catalog]: This master's current role is: LEADER
I20260929 15:52:12.251932 24977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f6ca6e85674452c9aaf391493fa4cac. Latest consensus state: current_term: 2 leader_uuid: "8f6ca6e85674452c9aaf391493fa4cac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6ca6e85674452c9aaf391493fa4cac" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36891 } } }
I20260929 15:52:12.252266 24977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac [sys.catalog]: This master's current role is: LEADER
I20260929 15:52:12.254648 24981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:52:12.261116 24981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:52:12.262598 24981 catalog_manager.cc:1259] Loaded cluster ID: 8f6f34f934bb476aabfde6d86473ae61
I20260929 15:52:12.262758 24981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:52:12.267105 24981 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:52:12.272482 24981 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8f6ca6e85674452c9aaf391493fa4cac: Loaded TSK: 0
I20260929 15:52:12.274801 24981 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 15:52:12.535346 24973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:12.535696 24973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:12.535846 24973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:12.543678 24973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:12.543848 24973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:12.544032 24973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:52:12.554289 24973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.193.keytab
--principal=oryx/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:45543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=36525
--tserver_master_addrs=127.15.196.254:36891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:12.555862 24973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:12.557325 24973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:12.574398 24998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:12.575827 24997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:12.575868 25000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:12.577536 24973 server_base.cc:1034] running on GCE node
I20260929 15:52:12.578263 24973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:12.579443 24973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:12.580715 24973 hybrid_clock.cc:648] HybridClock initialized: now 1790697132580659 us; error 52 us; skew 500 ppm
I20260929 15:52:12.581073 24973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:52:12 dist-test-slave-c51v krb5kdc[24385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697132, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:52:12.584411 24973 init.cc:377] Logged in from keytab as oryx/127.15.196.193@KRBTEST.COM (short username oryx)
I20260929 15:52:12.586102 24973 webserver.cc:460] Webserver started at http://127.15.196.193:36525/ using document root <none> and password file <none>
I20260929 15:52:12.586825 24973 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:12.586978 24973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:12.592653 24973 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 15:52:12.595589 25007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:12.597154 24973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:52:12.597339 24973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "19ba08e78b27499b84acdd11b73efb1a"
format_stamp: "Formatted at 2026-09-29 15:52:09 on dist-test-slave-c51v"
server_key: "fae3e1f0538bfe0e6a8520048b66e17d"
server_key_iv: "61d066d9da7e910cce678fb166b23c93"
server_key_version: "encryptionkey@0"
I20260929 15:52:12.598362 24973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:12.611169 24973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:12.615360 24973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:12.615597 24973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:12.616487 24973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:52:12.617751 24973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:12.617836 24973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:12.617916 24973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:12.617970 24973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:12.650507 24973 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:45543
I20260929 15:52:12.650594 25119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:45543 every 8 connection(s)
I20260929 15:52:12.651990 24973 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:52:12.660673 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 24973
I20260929 15:52:12.661288 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:44389
--local_ip_for_outbound_sockets=127.15.196.194
--tserver_master_addrs=127.15.196.254:36891
--webserver_port=46069
--webserver_interface=127.15.196.194
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.196.194
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.194.keytab with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 15:52:12 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:12 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 15:52:12.670301 25122 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:12.654872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:35433 (local address 127.15.196.254:36891)
0929 15:52:12.655583 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:12.655595 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:12.655967 (+ 372us) server_negotiation.cc:407] Connection header received
0929 15:52:12.658099 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:12.658112 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:12.658428 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:12.658730 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:12.660145 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:12.661483 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:12.663056 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:12.663768 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:12.663966 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:12.669373 (+ 5407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:35433: BlockingRecv error: recv got EOF from 127.15.196.193:35433 (error 108)
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":262,"threads_started":1}
I20260929 15:52:12.670329 25123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:12.654899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:35433)
0929 15:52:12.655775 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:12.655842 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:12.657587 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:12.658873 (+ 1286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:12.658881 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:12.659446 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:12.659973 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:12.659992 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:12.661677 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:12.661681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:12.662900 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:12.662912 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:12.664007 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:12.664014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:12.664289 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:12.664974 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:12.664997 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:12.669093 (+ 4096us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":412,"threads_started":1}
W20260929 15:52:12.671060 25120 heartbeater.cc:644] Failed to heartbeat to 127.15.196.254:36891 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.196.254:36891: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Sep 29 15:52:12 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:12 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:12.679378 25123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:12.671681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:48665)
0929 15:52:12.671876 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:12.671896 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:12.672021 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:12.672332 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:12.672337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:12.672677 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:12.672808 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:12.672816 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:12.673817 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:12.673822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:12.674703 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:12.674713 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:12.675434 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:12.675439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:12.675613 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:12.676209 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:12.676226 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:12.679047 (+ 2821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260929 15:52:12.679517 25122 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:12.671867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:48665 (local address 127.15.196.254:36891)
0929 15:52:12.672042 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:12.672048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:12.672087 (+ 39us) server_negotiation.cc:407] Connection header received
0929 15:52:12.672146 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:12.672150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:12.672205 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:12.672306 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:12.672945 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:12.673677 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:12.674840 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:12.675230 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:12.675408 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:12.679256 (+ 3848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:48665: BlockingRecv error: recv got EOF from 127.15.196.193:48665 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 29 15:52:12 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:12 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:12.688911 25123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:12.680186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:44505)
0929 15:52:12.680439 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:12.680458 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:12.680549 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:12.680946 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:12.680949 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:12.681204 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:12.681336 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:12.681343 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:12.682914 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:12.682918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:12.683893 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:12.683904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:12.684834 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:12.684838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:12.684959 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:12.685565 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:12.685589 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:12.688612 (+ 3023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153}
W20260929 15:52:12.689081 25122 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:12.680410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:44505 (local address 127.15.196.254:36891)
0929 15:52:12.680601 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:12.680609 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:12.680629 (+ 20us) server_negotiation.cc:407] Connection header received
0929 15:52:12.680700 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:12.680706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:12.680805 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:12.680926 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:12.681471 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:12.682749 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:12.684034 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:12.684694 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:12.684819 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:12.688827 (+ 4008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:44505: BlockingRecv error: recv got EOF from 127.15.196.193:44505 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260929 15:52:12.689451 25120 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 15:52:13.006845 25124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:13.007236 25124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:13.007418 25124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:13.014712 25124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:13.014876 25124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:13.015062 25124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:52:13.025058 25124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.194.keytab
--principal=oryx/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:44389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=46069
--tserver_master_addrs=127.15.196.254:36891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:13.026527 25124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:13.027894 25124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:13.041759 25129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:13.041805 25130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:13.041833 25132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:13.042681 25124 server_base.cc:1034] running on GCE node
I20260929 15:52:13.043460 25124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:13.044495 25124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:13.045722 25124 hybrid_clock.cc:648] HybridClock initialized: now 1790697133045680 us; error 61 us; skew 500 ppm
I20260929 15:52:13.046037 25124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:52:13 dist-test-slave-c51v krb5kdc[24385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697133, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:52:13.049294 25124 init.cc:377] Logged in from keytab as oryx/127.15.196.194@KRBTEST.COM (short username oryx)
I20260929 15:52:13.050854 25124 webserver.cc:460] Webserver started at http://127.15.196.194:46069/ using document root <none> and password file <none>
I20260929 15:52:13.051649 25124 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:13.051806 25124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:13.057337 25124 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 15:52:13.060099 25139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:13.061543 25124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:52:13.061718 25124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "533240e3d3bb4870873f254be9a52e17"
format_stamp: "Formatted at 2026-09-29 15:52:10 on dist-test-slave-c51v"
server_key: "9b5c3d1e175742b3d071f2cbd510a5bc"
server_key_iv: "4c0d87cde61fdf2b9cc9ad44c7e27352"
server_key_version: "encryptionkey@0"
I20260929 15:52:13.062700 25124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:13.086796 25124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:13.091069 25124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:13.091377 25124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:13.092409 25124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:52:13.093772 25124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:13.093899 25124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:13.094034 25124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:13.094132 25124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:13.128994 25124 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:44389
I20260929 15:52:13.129106 25251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:44389 every 8 connection(s)
I20260929 15:52:13.130470 25124 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:52:13.135509 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 25124
I20260929 15:52:13.136395 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:42001
--local_ip_for_outbound_sockets=127.15.196.195
--tserver_master_addrs=127.15.196.254:36891
--webserver_port=37047
--webserver_interface=127.15.196.195
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.196.195
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.195.keytab with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 15:52:13 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:13 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 15:52:13.150331 25122 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:13.133131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:47633 (local address 127.15.196.254:36891)
0929 15:52:13.133318 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:13.133323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:13.134597 (+ 1274us) server_negotiation.cc:407] Connection header received
0929 15:52:13.137626 (+ 3029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:13.137645 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:13.137721 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:13.137820 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:13.140095 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.140926 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.143748 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.144590 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.144743 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:13.150000 (+ 5257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:47633: BlockingRecv error: recv got EOF from 127.15.196.194:47633 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 15:52:13.150801 25254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:13.133660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:47633)
0929 15:52:13.134380 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:13.134422 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:13.137286 (+ 2864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:13.138129 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:13.138143 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:13.138983 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:13.139887 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:13.139917 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.141159 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.141168 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:13.143568 (+ 2400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:13.143589 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.144810 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.144815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:13.145024 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:13.145902 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:13.145927 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:13.149692 (+ 3765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":272,"threads_started":1}
W20260929 15:52:13.151523 25252 heartbeater.cc:644] Failed to heartbeat to 127.15.196.254:36891 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.196.254:36891: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Sep 29 15:52:13 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:13 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 15:52:13.162045 25122 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:13.152260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:40567 (local address 127.15.196.254:36891)
0929 15:52:13.152479 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:13.152487 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:13.152509 (+ 22us) server_negotiation.cc:407] Connection header received
0929 15:52:13.152576 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:13.152584 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:13.152669 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:13.152789 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:13.153429 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.154764 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.156208 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.157108 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.157307 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:13.161851 (+ 4544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:40567: BlockingRecv error: recv got EOF from 127.15.196.194:40567 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 15:52:13.162071 25254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:13.152145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:40567)
0929 15:52:13.152328 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:13.152352 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:13.152434 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:13.152815 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:13.152818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:13.153136 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:13.153276 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:13.153284 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.154911 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.154915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:13.156024 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:13.156036 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.157286 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.157290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:13.157417 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:13.158102 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:13.158120 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:13.161658 (+ 3538us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
Sep 29 15:52:13 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:13 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:13.172659 25254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:13.162946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:39263)
0929 15:52:13.163127 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:13.163148 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:13.163259 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:13.163610 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:13.163615 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:13.163953 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:13.164111 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:13.164120 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.165353 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.165361 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:13.167081 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:13.167098 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.167960 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.167966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:13.168166 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:13.168877 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:13.168901 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:13.172328 (+ 3427us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260929 15:52:13.172791 25122 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:13.163064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:39263 (local address 127.15.196.254:36891)
0929 15:52:13.163251 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:13.163258 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:13.163278 (+ 20us) server_negotiation.cc:407] Connection header received
0929 15:52:13.163392 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:13.163398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:13.163471 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:13.163595 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:13.164256 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.165183 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.167253 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.167803 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.167908 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:13.172532 (+ 4624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:39263: BlockingRecv error: recv got EOF from 127.15.196.194:39263 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260929 15:52:13.173187 25252 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 15:52:13.505903 25255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:13.506269 25255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:13.506459 25255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:13.514582 25255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:13.514746 25255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:13.514954 25255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:52:13.525913 25255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/krb5kdc/oryx_127.15.196.195.keytab
--principal=oryx/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:42001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=37047
--tserver_master_addrs=127.15.196.254:36891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:13.527510 25255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:13.528920 25255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:13.543027 25261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:13.543129 25260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:13.543144 25263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:13.544521 25255 server_base.cc:1034] running on GCE node
I20260929 15:52:13.545256 25255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:13.546309 25255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:13.547574 25255 hybrid_clock.cc:648] HybridClock initialized: now 1790697133547482 us; error 75 us; skew 500 ppm
I20260929 15:52:13.547899 25255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:52:13 dist-test-slave-c51v krb5kdc[24385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697133, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:52:13.550968 25255 init.cc:377] Logged in from keytab as oryx/127.15.196.195@KRBTEST.COM (short username oryx)
I20260929 15:52:13.552623 25255 webserver.cc:460] Webserver started at http://127.15.196.195:37047/ using document root <none> and password file <none>
I20260929 15:52:13.553332 25255 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:13.553443 25255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:13.559355 25255 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 15:52:13.562081 25270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:13.563557 25255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:52:13.563740 25255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "f206f25ee4e641e1ba708f3424052dea"
format_stamp: "Formatted at 2026-09-29 15:52:10 on dist-test-slave-c51v"
server_key: "db3b6cdafedc1307f492d6b924e78afd"
server_key_iv: "b68444f162a7f92ebf75e5e765766bd9"
server_key_version: "encryptionkey@0"
I20260929 15:52:13.564726 25255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:13.618511 25255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:13.623188 25255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:13.623499 25255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:13.624565 25255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:52:13.625926 25255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:13.626056 25255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:13.626202 25255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:13.626291 25255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:13.664093 25255 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:42001
I20260929 15:52:13.664151 25382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:42001 every 8 connection(s)
I20260929 15:52:13.665681 25255 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:52:13.673601 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 25255
Sep 29 15:52:13 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:13 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 15:52:13.683008 25122 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:13.668414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:59599 (local address 127.15.196.254:36891)
0929 15:52:13.668595 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:13.668600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:13.669987 (+ 1387us) server_negotiation.cc:407] Connection header received
0929 15:52:13.671894 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:13.671899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:13.671965 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:13.672060 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:13.673594 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.674606 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.676677 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.677103 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.677239 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:13.682753 (+ 5514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:59599: BlockingRecv error: recv got EOF from 127.15.196.195:59599 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 15:52:13.683890 25385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:13.668943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:59599)
0929 15:52:13.669811 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:13.669865 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:13.671656 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:13.672309 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:13.672316 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:13.672828 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:13.673407 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:13.673425 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.674775 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.674780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:13.676526 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:13.676539 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.677335 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.677342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:13.677686 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:13.678384 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:13.678418 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:13.682557 (+ 4139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":364,"threads_started":1}
W20260929 15:52:13.684594 25383 heartbeater.cc:644] Failed to heartbeat to 127.15.196.254:36891 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.196.254:36891: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
I20260929 15:52:13.691898 25387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:13.674739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57042 (local address 127.15.196.254:36891)
0929 15:52:13.675263 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:13.675269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:13.675702 (+ 433us) server_negotiation.cc:407] Connection header received
0929 15:52:13.675842 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:13.675847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:13.675929 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:13.676034 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:13.677453 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.678521 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.680163 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.680890 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.681018 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:13.683934 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:13.683947 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:13.683970 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:13.684027 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:13.687782 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:13.688500 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:13.688510 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:13.688520 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:13.688759 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:13.689155 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:13.689166 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:13.689169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:13.689755 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:13.690075 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:13.690677 (+ 602us) server_negotiation.cc:299] Negotiation successful
0929 15:52:13.691206 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":273,"threads_started":1}
Sep 29 15:52:13 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:13 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 15:52:13.694975 25122 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:13.685394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:36107 (local address 127.15.196.254:36891)
0929 15:52:13.685586 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:13.685592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:13.685614 (+ 22us) server_negotiation.cc:407] Connection header received
0929 15:52:13.685731 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:13.685737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:13.685810 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:13.685933 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:13.686826 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.687660 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.689152 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.689641 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.689778 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:13.694726 (+ 4948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:36107: BlockingRecv error: recv got EOF from 127.15.196.195:36107 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 15:52:13.695621 25385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:13.685294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:36107)
0929 15:52:13.685476 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:13.685497 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:13.685604 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:13.686002 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:13.686009 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:13.686465 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:13.686674 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:13.686690 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.687795 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.687800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:13.688998 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:13.689009 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.689790 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.689797 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:13.689986 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:13.690621 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:13.690643 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:13.695310 (+ 4667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
Sep 29 15:52:13 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:13 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:13 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:13 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:13.704895 25388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:13.690064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:55985)
0929 15:52:13.694534 (+ 4470us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:13.694566 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:13.694705 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:13.695164 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:13.695170 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:13.696465 (+ 1295us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:13.696744 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:13.696757 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.698179 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.698184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:13.699536 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:13.699547 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.700496 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.700502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:13.700704 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:13.701391 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:13.701416 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:13.704545 (+ 3129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":4330,"thread_start_us":4196,"threads_started":1}
W20260929 15:52:13.705075 25387 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:13.691247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:55985 (local address 127.15.196.254:36891)
0929 15:52:13.692775 (+ 1528us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:13.692784 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:13.694658 (+ 1874us) server_negotiation.cc:407] Connection header received
0929 15:52:13.694852 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:13.694857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:13.694930 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:13.695034 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:13.696892 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.698036 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.699671 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.700306 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.700430 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:13.704745 (+ 4315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:55985: BlockingRecv error: recv got EOF from 127.15.196.193:55985 (error 108)
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":1333,"thread_start_us":1576,"threads_started":1}
W20260929 15:52:13.705554 25122 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:13.696372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:36981 (local address 127.15.196.254:36891)
0929 15:52:13.696707 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:13.696712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:13.696733 (+ 21us) server_negotiation.cc:407] Connection header received
0929 15:52:13.696767 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:13.696770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:13.696816 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:13.696924 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:13.697843 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.698932 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.700514 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.701117 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.701302 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:13.705373 (+ 4071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:36981: BlockingRecv error: recv got EOF from 127.15.196.195:36981 (error 108)
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":234}
I20260929 15:52:13.705955 25385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:13.696302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:36981)
0929 15:52:13.696454 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:13.696472 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:13.696567 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:13.697119 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:13.697124 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:13.697497 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:13.697707 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:13.697719 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.699064 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.699068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:13.700386 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:13.700401 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:13.701297 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:13.701302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:13.701427 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:13.702045 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:13.702063 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:13.705756 (+ 3693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 15:52:13.706257 25383 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 29 15:52:14 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:14 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:14.182407 25390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:14.173776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:33759)
0929 15:52:14.174374 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:14.174391 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:14.174503 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:14.174794 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:14.174797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:14.175257 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:14.175509 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:14.175519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:14.176469 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:14.176473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:14.177415 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:14.177426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:14.178090 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:14.178094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:14.178210 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:14.178925 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:14.178950 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:14.182185 (+ 3235us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":357,"threads_started":1}
W20260929 15:52:14.182662 25122 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:14.173821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:33759 (local address 127.15.196.254:36891)
0929 15:52:14.173983 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:14.173988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:14.174464 (+ 476us) server_negotiation.cc:407] Connection header received
0929 15:52:14.174601 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:14.174605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:14.174664 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:14.174745 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:14.175639 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:14.176350 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:14.177531 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:14.177924 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:14.178017 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:14.182439 (+ 4422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:33759: BlockingRecv error: recv got EOF from 127.15.196.194:33759 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 29 15:52:14 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:14 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:14 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:14 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:14.721424 25391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:14.706015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:38053)
0929 15:52:14.707407 (+ 1392us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:14.707430 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:14.707580 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:14.708154 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:14.708159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:14.708714 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:14.708948 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:14.708961 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:14.710182 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:14.710187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:14.711356 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:14.711373 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:14.712328 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:14.712334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:14.712498 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:14.717914 (+ 5416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:14.717934 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:14.721073 (+ 3139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1237,"thread_start_us":1126,"threads_started":1}
W20260929 15:52:14.721654 25392 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:14.706052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:38053 (local address 127.15.196.254:36891)
0929 15:52:14.706632 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:14.706638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:14.707705 (+ 1067us) server_negotiation.cc:407] Connection header received
0929 15:52:14.707771 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:14.707779 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:14.707864 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:14.707962 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:14.709090 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:14.710027 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:14.711493 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:14.712150 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:14.712292 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:14.721327 (+ 9035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:38053: BlockingRecv error: recv got EOF from 127.15.196.193:38053 (error 108)
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":426,"thread_start_us":286,"threads_started":1}
W20260929 15:52:14.722193 25394 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:14.706938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:45739 (local address 127.15.196.254:36891)
0929 15:52:14.707682 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:14.707689 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:14.708159 (+ 470us) server_negotiation.cc:407] Connection header received
0929 15:52:14.708316 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:14.708321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:14.708413 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:14.708539 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:14.709643 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:14.710672 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:14.712290 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:14.712947 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:14.713058 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:14.721930 (+ 8872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:45739: BlockingRecv error: recv got EOF from 127.15.196.195:45739 (error 108)
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":622,"thread_start_us":500,"threads_started":1}
I20260929 15:52:14.722618 25393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:14.706886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:45739)
0929 15:52:14.708039 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:14.708057 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:14.708187 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:14.708610 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:14.708615 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:14.709210 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:14.709468 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:14.709483 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:14.710811 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:14.710815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:14.712160 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:14.712173 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:14.713156 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:14.713161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:14.713292 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:14.717957 (+ 4665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:14.717974 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:14.722394 (+ 4420us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1042,"thread_start_us":931,"threads_started":1}
Sep 29 15:52:15 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:15 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:15.192835 25395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:15.183388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:46969)
0929 15:52:15.183918 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:15.183958 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:15.184140 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:15.184483 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:15.184488 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:15.185003 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:15.185212 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:15.185223 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:15.186199 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:15.186205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:15.187692 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:15.187707 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:15.188409 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:15.188415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:15.188586 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:15.189224 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:15.189248 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:15.192507 (+ 3259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":267,"threads_started":1}
W20260929 15:52:15.192996 25394 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:15.183584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:46969 (local address 127.15.196.254:36891)
0929 15:52:15.183766 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:15.183772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:15.184032 (+ 260us) server_negotiation.cc:407] Connection header received
0929 15:52:15.184281 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:15.184285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:15.184347 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:15.184428 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:15.185338 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:15.186064 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:15.187855 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:15.188257 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:15.188349 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:15.192749 (+ 4400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:46969: BlockingRecv error: recv got EOF from 127.15.196.194:46969 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 29 15:52:15 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:15 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:15 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:15.733094 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:15.722605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:51645)
0929 15:52:15.723238 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:15.723260 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:15.723423 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:15.724050 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:15.724056 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:15.724550 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:15.724783 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:15.724797 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:15.726319 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:15.726325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:15.727535 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:15.727552 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:15.728256 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:15.728262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:15.728451 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:15.729117 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:15.729135 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:15.732706 (+ 3571us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":340,"threads_started":1}
W20260929 15:52:15.733109 25397 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:15.722850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:51645 (local address 127.15.196.254:36891)
0929 15:52:15.723497 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:15.723503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:15.723521 (+ 18us) server_negotiation.cc:407] Connection header received
0929 15:52:15.723625 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:15.723630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:15.723715 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:15.723820 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:15.724954 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:15.726174 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:15.727679 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:15.728108 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:15.728199 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:15.732886 (+ 4687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:51645: BlockingRecv error: recv got EOF from 127.15.196.193:51645 (error 108)
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":851,"threads_started":1}
Sep 29 15:52:15 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:15.734434 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:15.723571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:43263)
0929 15:52:15.725914 (+ 2343us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:15.725933 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:15.726040 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:15.726644 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:15.726648 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:15.726990 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:15.727159 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:15.727167 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:15.728414 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:15.728418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:15.729310 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:15.729321 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:15.730246 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:15.730249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:15.730387 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:15.730976 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:15.731000 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:15.734088 (+ 3088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2231,"thread_start_us":2131,"threads_started":1}
W20260929 15:52:15.734575 25399 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:15.723624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:43263 (local address 127.15.196.254:36891)
0929 15:52:15.726319 (+ 2695us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:15.726325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:15.726341 (+ 16us) server_negotiation.cc:407] Connection header received
0929 15:52:15.726387 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:15.726392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:15.726469 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:15.726585 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:15.727346 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:15.728283 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:15.729456 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:15.730102 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:15.730246 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:15.734290 (+ 4044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:43263: BlockingRecv error: recv got EOF from 127.15.196.195:43263 (error 108)
Metrics: {"server-negotiator.queue_time_us":2567,"thread_start_us":2405,"threads_started":1}
Sep 29 15:52:16 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:16 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:16.202754 25400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:16.193899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:49525)
0929 15:52:16.194432 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:16.194450 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:16.194565 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:16.194862 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:16.194865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:16.195216 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:16.195457 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:16.195467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:16.196924 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:16.196931 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:16.197973 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:16.197987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:16.198635 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:16.198640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:16.198772 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:16.199474 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:16.199494 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:16.202391 (+ 2897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":296,"threads_started":1}
W20260929 15:52:16.202929 25399 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:16.194048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:49525 (local address 127.15.196.254:36891)
0929 15:52:16.194223 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:16.194228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:16.194525 (+ 297us) server_negotiation.cc:407] Connection header received
0929 15:52:16.194666 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:16.194669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:16.194728 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:16.194817 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:16.195636 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:16.196717 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:16.198106 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:16.198484 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:16.198582 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:16.202638 (+ 4056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:49525: BlockingRecv error: recv got EOF from 127.15.196.194:49525 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 29 15:52:16 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:16 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:16 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:16 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:16.744964 25401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:16.734151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:49915)
0929 15:52:16.734615 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:16.734632 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:16.734736 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:16.735406 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:16.735412 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:16.736027 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:16.736251 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:16.736282 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:16.737695 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:16.737702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:16.739000 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:16.739013 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:16.739997 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:16.740003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:16.740166 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:16.740936 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:16.740978 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:16.744585 (+ 3607us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":247,"threads_started":1}
W20260929 15:52:16.745090 25402 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:16.734163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:49915 (local address 127.15.196.254:36891)
0929 15:52:16.735068 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:16.735073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:16.735088 (+ 15us) server_negotiation.cc:407] Connection header received
0929 15:52:16.735138 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:16.735142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:16.735198 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:16.735336 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:16.736422 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:16.737537 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:16.739136 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:16.739799 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:16.739915 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:16.744781 (+ 4866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:49915: BlockingRecv error: recv got EOF from 127.15.196.193:49915 (error 108)
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":611,"threads_started":1}
I20260929 15:52:16.745522 25403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:16.735359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:40777)
0929 15:52:16.736076 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:16.736093 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:16.736213 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:16.736583 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:16.736589 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:16.736998 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:16.737206 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:16.737218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:16.738652 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:16.738656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:16.739905 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:16.739919 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:16.741017 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:16.741021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:16.741217 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:16.741781 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:16.741807 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:16.745076 (+ 3269us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":507,"threads_started":1}
W20260929 15:52:16.745589 25404 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:16.735397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:40777 (local address 127.15.196.254:36891)
0929 15:52:16.735916 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:16.735923 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:16.736156 (+ 233us) server_negotiation.cc:407] Connection header received
0929 15:52:16.736321 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:16.736328 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:16.736411 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:16.736503 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:16.737351 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:16.738521 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:16.740066 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:16.740639 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:16.740761 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:16.745276 (+ 4515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:40777: BlockingRecv error: recv got EOF from 127.15.196.195:40777 (error 108)
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":244,"threads_started":1}
Sep 29 15:52:17 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:17 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:17.212601 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:17.203953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:39655)
0929 15:52:17.204463 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:17.204486 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:17.204653 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:17.205292 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:17.205295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:17.205815 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:17.205978 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:17.205987 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:17.207021 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:17.207025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:17.208040 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:17.208053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:17.208715 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:17.208720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:17.208839 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:17.209503 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:17.209523 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:17.212369 (+ 2846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":466,"threads_started":1}
W20260929 15:52:17.212826 25404 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:17.204217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:39655 (local address 127.15.196.254:36891)
0929 15:52:17.204410 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:17.204416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:17.204564 (+ 148us) server_negotiation.cc:407] Connection header received
0929 15:52:17.204796 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:17.204802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:17.204891 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:17.205026 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:17.206118 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:17.206905 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:17.208189 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:17.208590 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:17.208685 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:17.212607 (+ 3922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:39655: BlockingRecv error: recv got EOF from 127.15.196.194:39655 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 29 15:52:17 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:17 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:17 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:17 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:17.757562 25406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:17.746265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:34087)
0929 15:52:17.746951 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:17.746980 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:17.747156 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:17.748100 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:17.748107 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:17.748609 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:17.748842 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:17.748856 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:17.750319 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:17.750326 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:17.751929 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:17.751955 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:17.752804 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:17.752810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:17.752990 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:17.753744 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:17.753771 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:17.757194 (+ 3423us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":437,"threads_started":1}
I20260929 15:52:17.757943 25408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:17.746470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:50809)
0929 15:52:17.747892 (+ 1422us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:17.747911 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:17.748040 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:17.748533 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:17.748537 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:17.748977 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:17.749137 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:17.749146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:17.750567 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:17.750574 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:17.752177 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:17.752194 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:17.753054 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:17.753060 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:17.753258 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:17.753954 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:17.753973 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:17.757568 (+ 3595us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1297,"thread_start_us":1211,"threads_started":1}
W20260929 15:52:17.757941 25407 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:17.746394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:34087 (local address 127.15.196.254:36891)
0929 15:52:17.747685 (+ 1291us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:17.747693 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:17.747716 (+ 23us) server_negotiation.cc:407] Connection header received
0929 15:52:17.747781 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:17.747786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:17.747892 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:17.748555 (+ 663us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:17.748991 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:17.750138 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:17.752072 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:17.752632 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:17.752791 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:17.757441 (+ 4650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:34087: BlockingRecv error: recv got EOF from 127.15.196.193:34087 (error 108)
Metrics: {"server-negotiator.queue_time_us":1159,"thread_start_us":1067,"threads_started":1}
W20260929 15:52:17.758320 25409 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:17.746572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:50809 (local address 127.15.196.254:36891)
0929 15:52:17.748258 (+ 1686us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:17.748262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:17.748278 (+ 16us) server_negotiation.cc:407] Connection header received
0929 15:52:17.748318 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:17.748321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:17.748366 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:17.748453 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:17.749298 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:17.750425 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:17.752346 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:17.752892 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:17.753033 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:17.757891 (+ 4858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:50809: BlockingRecv error: recv got EOF from 127.15.196.195:50809 (error 108)
Metrics: {"server-negotiator.queue_time_us":1600,"thread_start_us":1440,"threads_started":1}
Sep 29 15:52:18 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:18 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:18.222004 25410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:18.213703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:47727)
0929 15:52:18.214321 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:18.214339 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:18.214448 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:18.214813 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:18.214824 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:18.215212 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:18.215477 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:18.215487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:18.216431 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:18.216435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:18.217331 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:18.217342 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:18.217966 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:18.217970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:18.218084 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:18.218745 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:18.218762 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:18.221642 (+ 2880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":297,"threads_started":1}
W20260929 15:52:18.222060 25409 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:18.213974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:47727 (local address 127.15.196.254:36891)
0929 15:52:18.214159 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:18.214167 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:18.214428 (+ 261us) server_negotiation.cc:407] Connection header received
0929 15:52:18.214563 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:18.214568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:18.214647 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:18.214736 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:18.215605 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:18.216317 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:18.217449 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:18.217852 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:18.217940 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:18.221848 (+ 3908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:47727: BlockingRecv error: recv got EOF from 127.15.196.194:47727 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 15:52:18 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:18 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:18 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:18 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:18.769393 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:18.758578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:35117)
0929 15:52:18.759237 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:18.759261 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:18.759412 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:18.759829 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:18.759835 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:18.760367 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:18.760614 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:18.760629 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:18.761946 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:18.761951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:18.763147 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:18.763160 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:18.764191 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:18.764201 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:18.764430 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:18.765602 (+ 1172us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:18.765629 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:18.769055 (+ 3426us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":346,"threads_started":1}
W20260929 15:52:18.769601 25412 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:18.758666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:35117 (local address 127.15.196.254:36891)
0929 15:52:18.759498 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:18.759506 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:18.759528 (+ 22us) server_negotiation.cc:407] Connection header received
0929 15:52:18.759581 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:18.759587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:18.759659 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:18.759797 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:18.760773 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:18.761809 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:18.763308 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:18.763962 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:18.764099 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:18.769288 (+ 5189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:35117: BlockingRecv error: recv got EOF from 127.15.196.193:35117 (error 108)
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":548,"threads_started":1}
I20260929 15:52:18.769934 25413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:18.759279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:47369)
0929 15:52:18.759743 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:18.759765 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:18.759906 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:18.760641 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:18.760644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:18.761175 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:18.761365 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:18.761376 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:18.762715 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:18.762719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:18.763990 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:18.764006 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:18.765075 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:18.765082 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:18.765306 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:18.766085 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:18.766121 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:18.769442 (+ 3321us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":191,"threads_started":1}
W20260929 15:52:18.770052 25414 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:18.759647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:47369 (local address 127.15.196.254:36891)
0929 15:52:18.760337 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:18.760344 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:18.760360 (+ 16us) server_negotiation.cc:407] Connection header received
0929 15:52:18.760403 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:18.760407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:18.760484 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:18.760600 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:18.761521 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:18.762581 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:18.764145 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:18.764848 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:18.765050 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:18.769706 (+ 4656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:47369: BlockingRecv error: recv got EOF from 127.15.196.195:47369 (error 108)
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":447,"threads_started":1}
Sep 29 15:52:19 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:19 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:19.230895 25415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:19.222978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:39545)
0929 15:52:19.223563 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:19.223580 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:19.223711 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:19.223999 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:19.224002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:19.224375 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:19.224526 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:19.224534 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:19.225508 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:19.225512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:19.226400 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:19.226411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:19.227041 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:19.227045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:19.227169 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:19.227790 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:19.227808 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:19.230568 (+ 2760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":348,"threads_started":1}
W20260929 15:52:19.231026 25414 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:19.223077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:39545 (local address 127.15.196.254:36891)
0929 15:52:19.223253 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:19.223257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:19.223673 (+ 416us) server_negotiation.cc:407] Connection header received
0929 15:52:19.223810 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:19.223814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:19.223875 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:19.223954 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:19.224656 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:19.225387 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:19.226521 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:19.226905 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:19.226993 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:19.230783 (+ 3790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:39545: BlockingRecv error: recv got EOF from 127.15.196.194:39545 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 29 15:52:19 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:19 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:19 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:19 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:19.786633 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:19.770457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:58827)
0929 15:52:19.771095 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:19.771119 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:19.771254 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:19.772239 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:19.772245 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:19.772832 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:19.773060 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:19.773075 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:19.774294 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:19.774300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:19.775204 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:19.775216 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:19.775929 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:19.775935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:19.776111 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:19.783219 (+ 7108us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:19.783238 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:19.786276 (+ 3038us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":284,"threads_started":1}
W20260929 15:52:19.786890 25417 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:19.770584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:58827 (local address 127.15.196.254:36891)
0929 15:52:19.771839 (+ 1255us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:19.771845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:19.771862 (+ 17us) server_negotiation.cc:407] Connection header received
0929 15:52:19.771922 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:19.771929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:19.772027 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:19.773212 (+ 1185us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:19.773373 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:19.774144 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:19.775367 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:19.775781 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:19.775917 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:19.786495 (+ 10578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:58827: BlockingRecv error: recv got EOF from 127.15.196.193:58827 (error 108)
Metrics: {"server-negotiator.queue_time_us":1096,"thread_start_us":914,"threads_started":1}
I20260929 15:52:19.787267 25418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:19.770961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:39279)
0929 15:52:19.772243 (+ 1282us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:19.772259 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:19.772384 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:19.772710 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:19.772715 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:19.773179 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:19.773365 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:19.773376 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:19.774561 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:19.774566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:19.775537 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:19.775550 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:19.776306 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:19.776312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:19.776503 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:19.783186 (+ 6683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:19.783209 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:19.786912 (+ 3703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1171,"thread_start_us":1055,"threads_started":1}
W20260929 15:52:19.787583 25419 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:19.771208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:39279 (local address 127.15.196.254:36891)
0929 15:52:19.772204 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:19.772211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:19.772342 (+ 131us) server_negotiation.cc:407] Connection header received
0929 15:52:19.772498 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:19.772501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:19.772555 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:19.772649 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:19.773545 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:19.774371 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:19.775682 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:19.776145 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:19.776291 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:19.787238 (+ 10947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:39279: BlockingRecv error: recv got EOF from 127.15.196.195:39279 (error 108)
Metrics: {"server-negotiator.queue_time_us":882,"thread_start_us":613,"threads_started":1}
Sep 29 15:52:20 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:20 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:20.241434 25420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:20.232023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:48459)
0929 15:52:20.232674 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:20.232692 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:20.232824 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:20.233145 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:20.233148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:20.233525 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:20.233711 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:20.233720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:20.234770 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:20.234779 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:20.236256 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:20.236273 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:20.237148 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:20.237163 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:20.237297 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:20.238064 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:20.238088 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:20.241052 (+ 2964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":285,"threads_started":1}
W20260929 15:52:20.241480 25419 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:20.232155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:48459 (local address 127.15.196.254:36891)
0929 15:52:20.232354 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:20.232359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:20.232778 (+ 419us) server_negotiation.cc:407] Connection header received
0929 15:52:20.232935 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:20.232939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:20.233004 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:20.233097 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:20.233848 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:20.234600 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:20.236412 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:20.236944 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:20.237044 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:20.241255 (+ 4211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:48459: BlockingRecv error: recv got EOF from 127.15.196.194:48459 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Sep 29 15:52:20 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:20 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:20 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:20 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:20.798039 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:20.787650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:36529)
0929 15:52:20.788224 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:20.788247 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:20.788379 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:20.788921 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:20.788929 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:20.789513 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:20.789734 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:20.789750 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:20.791350 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:20.791356 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:20.792679 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:20.792691 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:20.793675 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:20.793682 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:20.793842 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:20.794559 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:20.794583 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:20.797702 (+ 3119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":264,"threads_started":1}
W20260929 15:52:20.798210 25422 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:20.787812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:36529 (local address 127.15.196.254:36891)
0929 15:52:20.788545 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:20.788551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:20.788569 (+ 18us) server_negotiation.cc:407] Connection header received
0929 15:52:20.788624 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:20.788630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:20.788716 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:20.788863 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:20.789883 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:20.790977 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:20.792833 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:20.793497 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:20.793611 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:20.797920 (+ 4309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:36529: BlockingRecv error: recv got EOF from 127.15.196.193:36529 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":614,"thread_start_us":481,"threads_started":1}
I20260929 15:52:20.798467 25423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:20.788149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:58347)
0929 15:52:20.789361 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:20.789381 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:20.789490 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:20.789870 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:20.789875 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:20.790396 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:20.790614 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:20.790625 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:20.792039 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:20.792051 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:20.793427 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:20.793443 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:20.794437 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:20.794446 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:20.794576 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:20.795134 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:20.795152 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:20.798135 (+ 2983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1065,"thread_start_us":795,"threads_started":1}
W20260929 15:52:20.798566 25424 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:20.788595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:58347 (local address 127.15.196.254:36891)
0929 15:52:20.789553 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:20.789559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:20.789578 (+ 19us) server_negotiation.cc:407] Connection header received
0929 15:52:20.789622 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:20.789626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:20.789698 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:20.789808 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:20.790782 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:20.791914 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:20.793601 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:20.794264 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:20.794415 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:20.798309 (+ 3894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:58347: BlockingRecv error: recv got EOF from 127.15.196.195:58347 (error 108)
Metrics: {"server-negotiator.queue_time_us":840,"thread_start_us":723,"threads_started":1}
Sep 29 15:52:21 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:21 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:21.250849 25425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:21.242474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:54997)
0929 15:52:21.243020 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:21.243038 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:21.243151 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:21.243503 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:21.243506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:21.243891 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:21.244114 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:21.244124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:21.245414 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:21.245422 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:21.246418 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:21.246432 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:21.247201 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:21.247206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:21.247367 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:21.247960 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:21.247979 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:21.250570 (+ 2591us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":288,"threads_started":1}
W20260929 15:52:21.251056 25424 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:21.242518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:54997 (local address 127.15.196.254:36891)
0929 15:52:21.242708 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:21.242713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:21.243120 (+ 407us) server_negotiation.cc:407] Connection header received
0929 15:52:21.243234 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:21.243237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:21.243346 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:21.243441 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:21.244243 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:21.245194 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:21.246566 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:21.247033 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:21.247136 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:21.250772 (+ 3636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:54997: BlockingRecv error: recv got EOF from 127.15.196.194:54997 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 29 15:52:21 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:21 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:21 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:21 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:21.812595 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:21.799413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:45239)
0929 15:52:21.803993 (+ 4580us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:21.804029 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:21.804167 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:21.804588 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:21.804592 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:21.804983 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:21.805161 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:21.805174 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:21.806256 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:21.806260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:21.807148 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:21.807158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:21.808275 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:21.808283 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:21.808457 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:21.809020 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:21.809045 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:21.812261 (+ 3216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":4409,"thread_start_us":4316,"threads_started":1}
W20260929 15:52:21.812765 25427 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:21.799411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:45239 (local address 127.15.196.254:36891)
0929 15:52:21.803785 (+ 4374us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:21.803791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:21.804130 (+ 339us) server_negotiation.cc:407] Connection header received
0929 15:52:21.804317 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:21.804322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:21.804433 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:21.804543 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:21.805308 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:21.806124 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:21.807344 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:21.808070 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:21.808543 (+ 473us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:21.812445 (+ 3902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:45239: BlockingRecv error: recv got EOF from 127.15.196.195:45239 (error 108)
Metrics: {"mutex_wait_us":354,"server-negotiator.queue_time_us":4245,"spinlock_wait_cycles":896,"thread_start_us":3449,"threads_started":1}
I20260929 15:52:21.812879 25426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:21.799288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:46095)
0929 15:52:21.803784 (+ 4496us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:21.803804 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:21.803957 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:21.804355 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:21.804360 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:21.804891 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:21.805052 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:21.805060 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:21.806177 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:21.806185 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:21.807124 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:21.807135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:21.808067 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:21.808073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:21.808256 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:21.809224 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:21.809247 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:21.812471 (+ 3224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":4328,"thread_start_us":3795,"threads_started":1}
W20260929 15:52:21.813102 25428 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:21.799391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:46095 (local address 127.15.196.254:36891)
0929 15:52:21.803155 (+ 3764us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:21.803161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:21.803901 (+ 740us) server_negotiation.cc:407] Connection header received
0929 15:52:21.804086 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:21.804093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:21.804186 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:21.804293 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:21.805199 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:21.806026 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:21.807302 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:21.807892 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:21.808054 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:21.812681 (+ 4627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:46095: BlockingRecv error: recv got EOF from 127.15.196.193:46095 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":3594,"spinlock_wait_cycles":38656,"thread_start_us":4136,"threads_started":1}
Sep 29 15:52:22 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:22 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:22.260829 25430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:22.251992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:44879)
0929 15:52:22.252535 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:22.252552 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:22.252663 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:22.253057 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:22.253061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:22.253449 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:22.253622 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:22.253631 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:22.254983 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:22.254992 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:22.256295 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:22.256310 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:22.257073 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:22.257078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:22.257197 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:22.257829 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:22.257848 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:22.260593 (+ 2745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":304,"threads_started":1}
W20260929 15:52:22.260994 25428 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:22.252024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:44879 (local address 127.15.196.254:36891)
0929 15:52:22.252200 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:22.252205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:22.252627 (+ 422us) server_negotiation.cc:407] Connection header received
0929 15:52:22.252761 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:22.252784 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:22.252878 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:22.252991 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:22.253760 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:22.254795 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:22.256433 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:22.256910 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:22.257005 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:22.260760 (+ 3755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:44879: BlockingRecv error: recv got EOF from 127.15.196.194:44879 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 29 15:52:22 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:22 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:22 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:22 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:22.823901 25431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:22.813977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:52857)
0929 15:52:22.815170 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:22.815192 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:22.815358 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:22.815727 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:22.815732 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:22.816133 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:22.816280 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:22.816290 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:22.817360 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:22.817366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:22.818293 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:22.818305 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:22.819038 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:22.819044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:22.819216 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:22.819975 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:22.819998 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:22.823569 (+ 3571us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1073,"thread_start_us":966,"threads_started":1}
W20260929 15:52:22.824047 25433 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:22.814101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:52857 (local address 127.15.196.254:36891)
0929 15:52:22.814593 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:22.814602 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:22.815317 (+ 715us) server_negotiation.cc:407] Connection header received
0929 15:52:22.815480 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:22.815484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:22.815548 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:22.815689 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:22.816432 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:22.817218 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:22.818433 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:22.818850 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:22.819005 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:22.823771 (+ 4766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:52857: BlockingRecv error: recv got EOF from 127.15.196.193:52857 (error 108)
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":243,"threads_started":1}
I20260929 15:52:22.824095 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:22.813992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:36997)
0929 15:52:22.815421 (+ 1429us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:22.815440 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:22.815564 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:22.815927 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:22.815934 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:22.816364 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:22.816517 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:22.816525 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:22.817619 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:22.817627 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:22.818557 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:22.818567 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:22.819356 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:22.819363 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:22.819565 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:22.820155 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:22.820177 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:22.823760 (+ 3583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1318,"thread_start_us":1158,"threads_started":1}
W20260929 15:52:22.824628 25434 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:22.814480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:36997 (local address 127.15.196.254:36891)
0929 15:52:22.814973 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:22.814980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:22.815508 (+ 528us) server_negotiation.cc:407] Connection header received
0929 15:52:22.815695 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:22.815701 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:22.815772 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:22.815878 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:22.816693 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:22.817492 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:22.818699 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:22.819169 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:22.819294 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:22.823967 (+ 4673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:36997: BlockingRecv error: recv got EOF from 127.15.196.195:36997 (error 108)
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":264,"threads_started":1}
Sep 29 15:52:23 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:23 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:23.270804 25435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:23.261863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:50031)
0929 15:52:23.262368 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:23.262385 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:23.262548 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:23.262923 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:23.262928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:23.263405 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:23.263560 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:23.263570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:23.264601 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:23.264612 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:23.265533 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:23.265544 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:23.266279 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:23.266285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:23.266448 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:23.267339 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:23.267368 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:23.270541 (+ 3173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":289,"threads_started":1}
W20260929 15:52:23.270936 25434 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:23.261938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:50031 (local address 127.15.196.254:36891)
0929 15:52:23.262173 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:23.262181 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:23.262478 (+ 297us) server_negotiation.cc:407] Connection header received
0929 15:52:23.262721 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:23.262725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:23.262785 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:23.262889 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:23.263706 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:23.264456 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:23.265674 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:23.266129 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:23.266263 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:23.270736 (+ 4473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:50031: BlockingRecv error: recv got EOF from 127.15.196.194:50031 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 29 15:52:23 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:23 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:23 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:23 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:23.835004 25436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:23.825020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:47335)
0929 15:52:23.826024 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:23.826047 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:23.826187 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:23.826733 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:23.826738 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:23.827272 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:23.827531 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:23.827545 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:23.828926 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:23.828932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:23.829828 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:23.829839 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:23.830621 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:23.830627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:23.830820 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:23.831513 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:23.831530 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:23.834671 (+ 3141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":859,"thread_start_us":735,"threads_started":1}
W20260929 15:52:23.835227 25439 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:23.825238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:47335 (local address 127.15.196.254:36891)
0929 15:52:23.826410 (+ 1172us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:23.826415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:23.826428 (+ 13us) server_negotiation.cc:407] Connection header received
0929 15:52:23.826490 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:23.826494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:23.826548 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:23.826642 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:23.827705 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:23.828798 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:23.829968 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:23.830469 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:23.830594 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:23.834871 (+ 4277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:47335: BlockingRecv error: recv got EOF from 127.15.196.193:47335 (error 108)
Metrics: {"mutex_wait_us":786,"server-negotiator.queue_time_us":1062,"thread_start_us":729,"threads_started":1}
I20260929 15:52:23.835258 25437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:23.825070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:49445)
0929 15:52:23.825771 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:23.825792 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:23.825915 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:23.826700 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:23.826706 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:23.827178 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:23.827413 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:23.827426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:23.828852 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:23.828858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:23.829788 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:23.829798 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:23.830539 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:23.830545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:23.830733 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:23.831596 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:23.831614 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:23.834944 (+ 3330us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":370,"threads_started":1}
W20260929 15:52:23.835588 25438 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:23.825236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:49445 (local address 127.15.196.254:36891)
0929 15:52:23.826310 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:23.826317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:23.826335 (+ 18us) server_negotiation.cc:407] Connection header received
0929 15:52:23.826410 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:23.826416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:23.826512 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:23.826628 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:23.827572 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:23.828716 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:23.829927 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:23.830384 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:23.830509 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:23.835159 (+ 4650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:49445: BlockingRecv error: recv got EOF from 127.15.196.195:49445 (error 108)
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":929,"thread_start_us":758,"threads_started":1}
Sep 29 15:52:24 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:24 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:24.280119 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:24.271810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:46865)
0929 15:52:24.272307 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:24.272324 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:24.272473 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:24.272802 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:24.272810 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:24.273245 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:24.273414 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:24.273423 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:24.274436 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:24.274444 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:24.275366 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:24.275378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:24.276045 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:24.276051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:24.276231 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:24.276827 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:24.276844 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:24.279771 (+ 2927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":274,"threads_started":1}
W20260929 15:52:24.280225 25438 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:24.271891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:46865 (local address 127.15.196.254:36891)
0929 15:52:24.272124 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:24.272133 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:24.272411 (+ 278us) server_negotiation.cc:407] Connection header received
0929 15:52:24.272596 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:24.272601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:24.272665 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:24.272765 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:24.273556 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:24.274302 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:24.275504 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:24.275897 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:24.276028 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:24.279965 (+ 3937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:46865: BlockingRecv error: recv got EOF from 127.15.196.194:46865 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 29 15:52:24 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:24 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:24 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:24 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:24.858033 25441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:24.836162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:35209)
0929 15:52:24.843146 (+ 6984us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:24.843172 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:24.843343 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:24.843840 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:24.843848 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:24.844405 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:24.844660 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:24.844716 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:24.845885 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:24.845891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:24.847219 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:24.847231 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:24.847945 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:24.847952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:24.848152 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:24.854108 (+ 5956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:24.854148 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:24.857670 (+ 3522us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":6831,"thread_start_us":6719,"threads_started":1}
I20260929 15:52:24.858047 25443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:24.836247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:34041)
0929 15:52:24.843785 (+ 7538us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:24.843804 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:24.844139 (+ 335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:24.844513 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:24.844516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:24.845295 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:24.845469 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:24.845478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:24.846988 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:24.846993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:24.847949 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:24.847966 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:24.848685 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:24.848690 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:24.848864 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:24.854009 (+ 5145us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:24.854033 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:24.857686 (+ 3653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":7430,"thread_start_us":7326,"threads_started":1}
W20260929 15:52:24.858253 25444 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:24.836231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:35209 (local address 127.15.196.254:36891)
0929 15:52:24.843350 (+ 7119us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:24.843360 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:24.843394 (+ 34us) server_negotiation.cc:407] Connection header received
0929 15:52:24.843474 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:24.843486 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:24.843610 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:24.843727 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:24.844829 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:24.845718 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:24.847393 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:24.847795 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:24.847929 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:24.857976 (+ 10047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:35209: BlockingRecv error: recv got EOF from 127.15.196.193:35209 (error 108)
Metrics: {"server-negotiator.queue_time_us":6891,"thread_start_us":7298,"threads_started":1}
W20260929 15:52:24.858831 25442 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:24.836280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:34041 (local address 127.15.196.254:36891)
0929 15:52:24.843508 (+ 7228us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:24.843520 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:24.844129 (+ 609us) server_negotiation.cc:407] Connection header received
0929 15:52:24.844268 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:24.844274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:24.844347 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:24.844627 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:24.845608 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:24.846839 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:24.848108 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:24.848520 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:24.848648 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:24.858560 (+ 9912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:34041: BlockingRecv error: recv got EOF from 127.15.196.195:34041 (error 108)
Metrics: {"server-negotiator.queue_time_us":7087,"thread_start_us":6332,"threads_started":1}
Sep 29 15:52:25 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:25 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:25.289424 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:25.281076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:59551)
0929 15:52:25.281648 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:25.281665 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:25.281777 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:25.282121 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:25.282124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:25.282484 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:25.282653 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:25.282661 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:25.283727 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:25.283731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:25.284640 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:25.284650 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:25.285258 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:25.285262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:25.285383 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:25.285957 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:25.285974 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:25.289086 (+ 3112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":325,"threads_started":1}
W20260929 15:52:25.289496 25442 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:25.281130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:59551 (local address 127.15.196.254:36891)
0929 15:52:25.281293 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:25.281298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:25.281736 (+ 438us) server_negotiation.cc:407] Connection header received
0929 15:52:25.281872 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:25.281875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:25.281964 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:25.282074 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:25.282767 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:25.283604 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:25.284756 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:25.285145 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:25.285233 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:25.289277 (+ 4044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:59551: BlockingRecv error: recv got EOF from 127.15.196.194:59551 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 29 15:52:25 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:25 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:25 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:25 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
W20260929 15:52:25.870352 25447 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:25.859185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:44469 (local address 127.15.196.254:36891)
0929 15:52:25.860437 (+ 1252us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:25.860443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:25.860461 (+ 18us) server_negotiation.cc:407] Connection header received
0929 15:52:25.860526 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:25.860533 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:25.860645 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:25.861166 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:25.862186 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:25.863035 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:25.864495 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:25.865154 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:25.865250 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:25.870000 (+ 4750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:44469: BlockingRecv error: recv got EOF from 127.15.196.195:44469 (error 108)
Metrics: {"server-negotiator.queue_time_us":1097,"thread_start_us":904,"threads_started":1}
I20260929 15:52:25.870424 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:25.859054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:44469)
0929 15:52:25.859686 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:25.859707 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:25.859838 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:25.861224 (+ 1386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:25.861228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:25.861858 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:25.862033 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:25.862043 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:25.863175 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:25.863181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:25.864361 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:25.864374 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:25.865293 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:25.865299 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:25.865509 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:25.866261 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:25.866279 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:25.870109 (+ 3830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":366,"threads_started":1}
I20260929 15:52:25.870678 25448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:25.859515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:57341)
0929 15:52:25.861237 (+ 1722us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:25.861256 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:25.861382 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:25.861719 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:25.861723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:25.862069 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:25.862305 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:25.862316 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:25.863838 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:25.863842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:25.864984 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:25.865001 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:25.866031 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:25.866040 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:25.866316 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:25.866886 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:25.866904 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:25.870349 (+ 3445us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1584,"thread_start_us":1024,"threads_started":1}
W20260929 15:52:25.871236 25449 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:25.859777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:57341 (local address 127.15.196.254:36891)
0929 15:52:25.860732 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:25.860738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:25.861358 (+ 620us) server_negotiation.cc:407] Connection header received
0929 15:52:25.861474 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:25.861483 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:25.861570 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:25.861697 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:25.862614 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:25.863706 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:25.865143 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:25.865847 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:25.865935 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:25.871048 (+ 5113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:57341: BlockingRecv error: recv got EOF from 127.15.196.193:57341 (error 108)
Metrics: {"server-negotiator.queue_time_us":808,"thread_start_us":606,"threads_started":1}
Sep 29 15:52:26 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:26 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:26.299647 25450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:26.290621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:54951)
0929 15:52:26.291267 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:26.291306 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:26.291473 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:26.291814 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:26.291820 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:26.292368 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:26.292570 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:26.292579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:26.293939 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:26.293944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:26.295103 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:26.295116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:26.295868 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:26.295873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:26.295992 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:26.296586 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:26.296604 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:26.299403 (+ 2799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":370,"threads_started":1}
W20260929 15:52:26.299834 25449 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:26.290860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:54951 (local address 127.15.196.254:36891)
0929 15:52:26.291113 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:26.291120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:26.291395 (+ 275us) server_negotiation.cc:407] Connection header received
0929 15:52:26.291610 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:26.291613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:26.291676 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:26.291778 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:26.292768 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:26.293793 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:26.295254 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:26.295698 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:26.295801 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:26.299593 (+ 3792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:54951: BlockingRecv error: recv got EOF from 127.15.196.194:54951 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Sep 29 15:52:26 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:26 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:26 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:26 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:26.894240 25451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:26.871417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:39841)
0929 15:52:26.885073 (+ 13656us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:26.885111 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:26.885286 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:26.885917 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:26.885923 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:26.886434 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:26.886596 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:26.886607 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:26.887730 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:26.887736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:26.888695 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:26.888706 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:26.889506 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:26.889513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:26.889685 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:26.890346 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:26.890375 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:26.893892 (+ 3517us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":13473,"thread_start_us":13433,"threads_started":1}
W20260929 15:52:26.894379 25454 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:26.871502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:39841 (local address 127.15.196.254:36891)
0929 15:52:26.884818 (+ 13316us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:26.884828 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:26.885406 (+ 578us) server_negotiation.cc:407] Connection header received
0929 15:52:26.885478 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:26.885485 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:26.885587 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:26.885702 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:26.886753 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:26.887586 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:26.888838 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:26.889331 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:26.889426 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:26.894102 (+ 4676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:39841: BlockingRecv error: recv got EOF from 127.15.196.195:39841 (error 108)
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":13125,"thread_start_us":14044,"threads_started":1}
I20260929 15:52:26.894620 25453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:26.871659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:60931)
0929 15:52:26.885480 (+ 13821us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:26.885505 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:26.885643 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:26.886107 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:26.886110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:26.886465 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:26.886636 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:26.886645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:26.887888 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:26.887892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:26.888801 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:26.888817 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:26.889657 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:26.889662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:26.889780 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:26.890538 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:26.890558 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:26.894281 (+ 3723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":13667,"thread_start_us":13554,"threads_started":1}
W20260929 15:52:26.894879 25452 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:26.871663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:60931 (local address 127.15.196.254:36891)
0929 15:52:26.885833 (+ 14170us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:26.885839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:26.885856 (+ 17us) server_negotiation.cc:407] Connection header received
0929 15:52:26.885896 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:26.885900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:26.885963 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:26.886067 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:26.886804 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:26.887724 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:26.888946 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:26.889508 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:26.889629 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:26.894491 (+ 4862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:60931: BlockingRecv error: recv got EOF from 127.15.196.193:60931 (error 108)
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":14067,"thread_start_us":12735,"threads_started":1}
Sep 29 15:52:27 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:27 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:27.309161 25455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:27.300530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:41385)
0929 15:52:27.301035 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:27.301054 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:27.301177 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:27.301489 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:27.301492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:27.301990 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:27.302173 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:27.302183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:27.303216 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:27.303220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:27.304259 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:27.304272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:27.304873 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:27.304877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:27.305036 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:27.305644 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:27.305662 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:27.308848 (+ 3186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":268,"threads_started":1}
W20260929 15:52:27.309240 25452 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:27.300617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:41385 (local address 127.15.196.254:36891)
0929 15:52:27.300779 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:27.300784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:27.301143 (+ 359us) server_negotiation.cc:407] Connection header received
0929 15:52:27.301296 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:27.301300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:27.301362 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:27.301440 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:27.302313 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:27.303080 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:27.304382 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:27.304754 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:27.304841 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:27.309044 (+ 4203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:41385: BlockingRecv error: recv got EOF from 127.15.196.194:41385 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 29 15:52:27 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:27 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:27 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.193@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:27 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.195@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:27.906744 25458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:27.895882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.193:36961)
0929 15:52:27.896999 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:27.897014 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:27.897166 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:27.897479 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:27.897485 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:27.897966 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:27.898134 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:27.898144 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:27.899492 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:27.899498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:27.900949 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:27.900963 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:27.901924 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:27.901931 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:27.902117 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:27.903074 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:27.903101 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:27.906462 (+ 3361us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1016,"thread_start_us":900,"threads_started":1}
W20260929 15:52:27.907016 25457 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:27.896194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:36961 (local address 127.15.196.254:36891)
0929 15:52:27.896867 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:27.896873 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:27.897126 (+ 253us) server_negotiation.cc:407] Connection header received
0929 15:52:27.897279 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:27.897282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:27.897340 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:27.897414 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:27.898257 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:27.899378 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:27.901059 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:27.901752 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:27.901903 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:27.906696 (+ 4793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.193:36961: BlockingRecv error: recv got EOF from 127.15.196.193:36961 (error 108)
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":537,"thread_start_us":266,"threads_started":1}
I20260929 15:52:27.907094 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:27.895480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.195:46179)
0929 15:52:27.895951 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:27.895973 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:27.896133 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:27.897126 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:27.897130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:27.897595 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:27.897841 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:27.897853 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:27.899331 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:27.899338 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:27.900815 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:27.900830 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:27.901845 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:27.901851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:27.902077 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:27.902903 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:27.902925 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:27.906777 (+ 3852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":234,"threads_started":1}
W20260929 15:52:27.907397 25459 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:27.895533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:46179 (local address 127.15.196.254:36891)
0929 15:52:27.896798 (+ 1265us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:27.896803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:27.896820 (+ 17us) server_negotiation.cc:407] Connection header received
0929 15:52:27.896875 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:27.896880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:27.896958 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:27.897053 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:27.897980 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:27.899125 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:27.900966 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:27.901673 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:27.901803 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:27.907052 (+ 5249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.195:46179: BlockingRecv error: recv got EOF from 127.15.196.195:46179 (error 108)
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":1101,"thread_start_us":1042,"threads_started":1}
Sep 29 15:52:28 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
Sep 29 15:52:28 dist-test-slave-c51v krb5kdc[24385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.196.194@KRBTEST.COM for oryx/127.15.196.254@KRBTEST.COM, Server not found in Kerberos database
I20260929 15:52:28.318780 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:28.310390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:36891 (local address 127.15.196.194:50999)
0929 15:52:28.310901 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:28.310918 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:28.311029 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:28.311390 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:28.311394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:28.311774 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:28.311939 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:28.311948 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:28.312911 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:28.312914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:28.313933 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:28.313945 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:28.314558 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:28.314562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:28.314670 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:28.315228 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:28.315245 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:28.318438 (+ 3193us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.196.254:36891: Server oryx/127.15.196.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":282,"threads_started":1}
W20260929 15:52:28.318861 25459 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:28.310630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:50999 (local address 127.15.196.254:36891)
0929 15:52:28.310838 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:28.310847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:28.310988 (+ 141us) server_negotiation.cc:407] Connection header received
0929 15:52:28.311142 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:28.311147 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:28.311208 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:28.311313 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:28.312061 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:28.312795 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:28.314059 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:28.314438 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:28.314531 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:28.318678 (+ 4147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.196.194:50999: BlockingRecv error: recv got EOF from 127.15.196.194:50999 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 15:52:28.682793 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 24973
I20260929 15:52:28.695901 25462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:28.684129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47408 (local address 127.15.196.193:45543)
0929 15:52:28.684806 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:28.684813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:28.684840 (+ 27us) server_negotiation.cc:407] Connection header received
0929 15:52:28.684898 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:28.684905 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:28.685132 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:28.685287 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:28.687063 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:28.688144 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:28.689699 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:28.690255 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:28.690360 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:28.693015 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:28.693024 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:28.693039 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:28.693072 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:28.695420 (+ 2348us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.196.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0929 15:52:28.695678 (+ 258us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47408: Request ticket server kudu/127.15.196.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":385,"threads_started":1}
W20260929 15:52:28.696022 25462 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47408: Request ticket server kudu/127.15.196.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260929 15:52:28.696053 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 24973: Not authorized: Client connection negotiation failed: client connection to 127.15.196.193:45543: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.196.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260929 15:52:28.696306 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 24973
I20260929 15:52:28.713531 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 25124
I20260929 15:52:28.725363 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:28.714510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35136 (local address 127.15.196.194:44389)
0929 15:52:28.715105 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:28.715112 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:28.715150 (+ 38us) server_negotiation.cc:407] Connection header received
0929 15:52:28.715212 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:28.715220 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:28.715504 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:28.715665 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:28.717000 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:28.718055 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:28.719153 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:28.719798 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:28.719896 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:28.722316 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:28.722323 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:28.722337 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:28.722368 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:28.724882 (+ 2514us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.196.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0929 15:52:28.725127 (+ 245us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35136: Request ticket server kudu/127.15.196.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":294,"threads_started":1}
W20260929 15:52:28.725476 25463 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35136: Request ticket server kudu/127.15.196.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260929 15:52:28.725530 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 25124: Not authorized: Client connection negotiation failed: client connection to 127.15.196.194:44389: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.196.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260929 15:52:28.725713 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 25124
I20260929 15:52:28.742519 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 25255
I20260929 15:52:28.754472 25464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:28.743656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38520 (local address 127.15.196.195:42001)
0929 15:52:28.744162 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:28.744169 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:28.744193 (+ 24us) server_negotiation.cc:407] Connection header received
0929 15:52:28.744254 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:28.744260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:28.744486 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:28.744684 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:28.746005 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:28.747009 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:28.748248 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:28.748798 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:28.748945 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:28.751439 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:28.751448 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:28.751464 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:28.751498 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:28.753817 (+ 2319us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.196.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0929 15:52:28.754146 (+ 329us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38520: Request ticket server kudu/127.15.196.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":237,"threads_started":1}
W20260929 15:52:28.754521 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 25255: Not authorized: Client connection negotiation failed: client connection to 127.15.196.195:42001: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.196.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260929 15:52:28.754585 25464 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38520: Request ticket server kudu/127.15.196.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260929 15:52:28.754685 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 25255
I20260929 15:52:28.771042 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 24904
I20260929 15:52:28.772363 24966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:28.890937 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 24904
2026-09-29T15:52:28Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21237 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-29T15:52:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:52:28Z Disabled control of system clock
I20260929 15:52:28.972820 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:33439
--webserver_interface=127.15.196.254
--webserver_port=0
--builtin_ntp_servers=127.15.196.212:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:33439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 15:52:29.309751 25469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:29.310107 25469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:29.310288 25469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:29.318138 25469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:52:29.318286 25469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:29.318358 25469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:29.318463 25469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:52:29.318549 25469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:52:29.328570 25469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:33439
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:33439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:29.330016 25469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:29.332444 25469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:29.345942 25474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:29.345877 25477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:29.345886 25475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:29.346390 25469 server_base.cc:1034] running on GCE node
I20260929 15:52:29.347555 25469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:29.349017 25469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:29.350234 25469 hybrid_clock.cc:648] HybridClock initialized: now 1790697149350165 us; error 68 us; skew 500 ppm
I20260929 15:52:29.350561 25469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:52:29.352955 25469 webserver.cc:460] Webserver started at http://127.15.196.254:40185/ using document root <none> and password file <none>
I20260929 15:52:29.353731 25469 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:29.353819 25469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:29.354099 25469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:29.357086 25469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "4ba7c720f2d94976a4ac53a09db134b3"
format_stamp: "Formatted at 2026-09-29 15:52:29 on dist-test-slave-c51v"
server_key: "c83ee39d3c452ba99299f8e4216f82c3"
server_key_iv: "d4fd77c3d824b60850832788f8223408"
server_key_version: "encryptionkey@0"
I20260929 15:52:29.357857 25469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "4ba7c720f2d94976a4ac53a09db134b3"
format_stamp: "Formatted at 2026-09-29 15:52:29 on dist-test-slave-c51v"
server_key: "c83ee39d3c452ba99299f8e4216f82c3"
server_key_iv: "d4fd77c3d824b60850832788f8223408"
server_key_version: "encryptionkey@0"
I20260929 15:52:29.362979 25469 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 15:52:29.366468 25483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:29.367897 25469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:52:29.368039 25469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "4ba7c720f2d94976a4ac53a09db134b3"
format_stamp: "Formatted at 2026-09-29 15:52:29 on dist-test-slave-c51v"
server_key: "c83ee39d3c452ba99299f8e4216f82c3"
server_key_iv: "d4fd77c3d824b60850832788f8223408"
server_key_version: "encryptionkey@0"
I20260929 15:52:29.368224 25469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:29.388762 25469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:29.389565 25469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:29.389801 25469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:29.408125 25469 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:33439
I20260929 15:52:29.408135 25534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:33439 every 8 connection(s)
I20260929 15:52:29.409510 25469 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:52:29.413357 25535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:29.414820 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 25469
I20260929 15:52:29.415022 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:52:29.415920 16147 external_mini_cluster.cc:1428] Setting key e214c9b7166f0183b8b3d2ce0b45a8e9
I20260929 15:52:29.428871 25535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3: Bootstrap starting.
I20260929 15:52:29.429950 25538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:29.418436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53098 (local address 127.15.196.254:33439)
0929 15:52:29.419547 (+ 1111us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:29.419558 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:29.419603 (+ 45us) server_negotiation.cc:407] Connection header received
0929 15:52:29.421199 (+ 1596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:29.421233 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:29.421890 (+ 657us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:29.422390 (+ 500us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:29.422943 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:29.424259 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:29.425552 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:29.426256 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:29.426460 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:29.427659 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:29.427667 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:29.427682 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:52:29.428259 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:29.428335 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:52:29.428355 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:29.428492 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:29.428690 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 15:52:29.429006 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":335,"threads_started":1}
I20260929 15:52:29.432735 25535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:29.433903 25535 log.cc:824] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3: Log is configured to *not* fsync() on all Append() calls
I20260929 15:52:29.437142 25535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3: No bootstrap required, opened a new log
I20260929 15:52:29.446427 25535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba7c720f2d94976a4ac53a09db134b3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 33439 } }
I20260929 15:52:29.446842 25535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:29.446934 25535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ba7c720f2d94976a4ac53a09db134b3, State: Initialized, Role: FOLLOWER
I20260929 15:52:29.447600 25535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba7c720f2d94976a4ac53a09db134b3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 33439 } }
I20260929 15:52:29.447831 25535 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:52:29.447958 25535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:52:29.448112 25535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:29.450652 25535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba7c720f2d94976a4ac53a09db134b3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 33439 } }
I20260929 15:52:29.451186 25535 leader_election.cc:302] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ba7c720f2d94976a4ac53a09db134b3; no voters:
I20260929 15:52:29.452119 25535 leader_election.cc:288] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:52:29.452172 25540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:52:29.452852 25540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [term 1 LEADER]: Becoming Leader. State: Replica: 4ba7c720f2d94976a4ac53a09db134b3, State: Running, Role: LEADER
I20260929 15:52:29.453501 25540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba7c720f2d94976a4ac53a09db134b3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 33439 } }
I20260929 15:52:29.455082 25535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:52:29.456337 25542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ba7c720f2d94976a4ac53a09db134b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba7c720f2d94976a4ac53a09db134b3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 33439 } } }
I20260929 15:52:29.456815 25542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [sys.catalog]: This master's current role is: LEADER
I20260929 15:52:29.456367 25541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ba7c720f2d94976a4ac53a09db134b3. Latest consensus state: current_term: 1 leader_uuid: "4ba7c720f2d94976a4ac53a09db134b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ba7c720f2d94976a4ac53a09db134b3" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 33439 } } }
I20260929 15:52:29.457283 25541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3 [sys.catalog]: This master's current role is: LEADER
I20260929 15:52:29.459229 25545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:52:29.464362 25545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:52:29.473492 25545 catalog_manager.cc:1347] Generated new cluster ID: 23a927289c0b405db76087718660cb91
I20260929 15:52:29.473639 25545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:52:29.494158 25545 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:52:29.495630 25545 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:52:29.519479 25545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ba7c720f2d94976a4ac53a09db134b3: Generated new TSK 0
I20260929 15:52:29.520505 25545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:52:29.529558 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:33439
--builtin_ntp_servers=127.15.196.212:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 15:52:29.860005 25559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:29.860348 25559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:29.860476 25559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:29.867928 25559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:29.868036 25559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:29.868140 25559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:52:29.877538 25559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:33439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:29.878857 25559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:29.880717 25559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:29.893819 25567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:29.893869 25565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:29.893776 25564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:29.900969 25559 server_base.cc:1034] running on GCE node
I20260929 15:52:29.901683 25559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:29.902725 25559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:29.903950 25559 hybrid_clock.cc:648] HybridClock initialized: now 1790697149903905 us; error 47 us; skew 500 ppm
I20260929 15:52:29.904289 25559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:52:29.912461 25559 webserver.cc:460] Webserver started at http://127.15.196.193:43975/ using document root <none> and password file <none>
I20260929 15:52:29.913261 25559 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:29.913394 25559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:29.913734 25559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:29.916811 25559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "9a7e6d6eca5b48668f23e6f29734bc8a"
format_stamp: "Formatted at 2026-09-29 15:52:29 on dist-test-slave-c51v"
server_key: "15962c573648628c2a224d986d8713be"
server_key_iv: "ff91b3311f59bbdfeb6202067bccbf7d"
server_key_version: "encryptionkey@0"
I20260929 15:52:29.917621 25559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "9a7e6d6eca5b48668f23e6f29734bc8a"
format_stamp: "Formatted at 2026-09-29 15:52:29 on dist-test-slave-c51v"
server_key: "15962c573648628c2a224d986d8713be"
server_key_iv: "ff91b3311f59bbdfeb6202067bccbf7d"
server_key_version: "encryptionkey@0"
I20260929 15:52:29.922984 25559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 15:52:29.926348 25573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:29.927729 25559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:52:29.927902 25559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "9a7e6d6eca5b48668f23e6f29734bc8a"
format_stamp: "Formatted at 2026-09-29 15:52:29 on dist-test-slave-c51v"
server_key: "15962c573648628c2a224d986d8713be"
server_key_iv: "ff91b3311f59bbdfeb6202067bccbf7d"
server_key_version: "encryptionkey@0"
I20260929 15:52:29.928153 25559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:29.946524 25559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:29.947460 25559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:29.947751 25559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:29.948678 25559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:52:29.950063 25559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:29.950196 25559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:29.950311 25559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:29.950397 25559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:29.983038 25559 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:44739
I20260929 15:52:29.983091 25685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:44739 every 8 connection(s)
I20260929 15:52:29.984387 25559 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:52:29.992700 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 25559
I20260929 15:52:29.992908 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:52:29.993727 16147 external_mini_cluster.cc:1428] Setting key 3fbc067d1c6248a6000867b247ad3994
I20260929 15:52:29.996744 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:33439
--builtin_ntp_servers=127.15.196.212:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 15:52:29.997376 25688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:29.986840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:36721 (local address 127.15.196.254:33439)
0929 15:52:29.987278 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:29.987304 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:29.988479 (+ 1175us) server_negotiation.cc:407] Connection header received
0929 15:52:29.990158 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:29.990163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:29.990231 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:29.990328 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:29.991384 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:29.992085 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:29.994488 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:29.994928 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:29.995058 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:29.996547 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:29.996553 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:29.996556 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:52:29.996573 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:29.996612 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:52:29.996624 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:29.996714 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:29.997040 (+ 326us) server_negotiation.cc:299] Negotiation successful
0929 15:52:29.997115 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":198,"threads_started":1}
I20260929 15:52:29.998831 25689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:29.987451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:33439 (local address 127.15.196.193:36721)
0929 15:52:29.988314 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:29.988363 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:29.989906 (+ 1543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:29.990525 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:29.990533 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:29.990615 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:29.991182 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:29.991199 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:29.992245 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:29.992254 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:29.994341 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:29.994358 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:29.995172 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:29.995180 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:29.995478 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:29.996196 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:52:29.996199 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:29.996259 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:52:29.996357 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:29.996744 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:29.996757 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:29.996768 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 15:52:29.996967 (+ 199us) client_negotiation.cc:240] Negotiation successful
0929 15:52:29.997406 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":250,"threads_started":1}
I20260929 15:52:30.000389 25686 heartbeater.cc:347] Connected to a master server at 127.15.196.254:33439
I20260929 15:52:30.000706 25686 heartbeater.cc:464] Registering TS with master...
I20260929 15:52:30.001472 25686 heartbeater.cc:511] Master 127.15.196.254:33439 requested a full tablet report, sending...
I20260929 15:52:30.003470 25500 ts_manager.cc:194] Registered new tserver with Master: 9a7e6d6eca5b48668f23e6f29734bc8a (127.15.196.193:44739)
I20260929 15:52:30.004920 25500 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.196.193:36721
W20260929 15:52:30.324812 25690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:30.325106 25690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:30.325258 25690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:30.332681 25690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:30.332787 25690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:30.332939 25690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:52:30.343220 25690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:33439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:30.344626 25690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:30.346388 25690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:30.359390 25695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:30.359431 25698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:30.359396 25696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:30.360462 25690 server_base.cc:1034] running on GCE node
I20260929 15:52:30.361016 25690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:30.361953 25690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:30.363154 25690 hybrid_clock.cc:648] HybridClock initialized: now 1790697150363116 us; error 28 us; skew 500 ppm
I20260929 15:52:30.363493 25690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:52:30.365643 25690 webserver.cc:460] Webserver started at http://127.15.196.194:40861/ using document root <none> and password file <none>
I20260929 15:52:30.366331 25690 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:30.366447 25690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:30.366768 25690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:30.369658 25690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "a57059a5b53d4ad89b5c5a0cb73b567a"
format_stamp: "Formatted at 2026-09-29 15:52:30 on dist-test-slave-c51v"
server_key: "d9c1097d38010501451eadaa8b58025f"
server_key_iv: "d76f787d27e075c2ea7ab91404825b67"
server_key_version: "encryptionkey@0"
I20260929 15:52:30.370435 25690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "a57059a5b53d4ad89b5c5a0cb73b567a"
format_stamp: "Formatted at 2026-09-29 15:52:30 on dist-test-slave-c51v"
server_key: "d9c1097d38010501451eadaa8b58025f"
server_key_iv: "d76f787d27e075c2ea7ab91404825b67"
server_key_version: "encryptionkey@0"
I20260929 15:52:30.375808 25690 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 15:52:30.379422 25704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:30.380797 25690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:52:30.380970 25690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "a57059a5b53d4ad89b5c5a0cb73b567a"
format_stamp: "Formatted at 2026-09-29 15:52:30 on dist-test-slave-c51v"
server_key: "d9c1097d38010501451eadaa8b58025f"
server_key_iv: "d76f787d27e075c2ea7ab91404825b67"
server_key_version: "encryptionkey@0"
I20260929 15:52:30.381194 25690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:30.411054 25690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:30.412168 25690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:30.412487 25690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:30.413501 25690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:52:30.414997 25690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:30.415123 25690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:30.415256 25690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:30.415372 25690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260929 15:52:30.448983 25690 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:44839
I20260929 15:52:30.449043 25816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:44839 every 8 connection(s)
I20260929 15:52:30.450320 25690 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:52:30.450765 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 25690
I20260929 15:52:30.450937 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:52:30.451602 16147 external_mini_cluster.cc:1428] Setting key f3eb2357122b2f2b6f348780a1722875
I20260929 15:52:30.454870 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:33439
--builtin_ntp_servers=127.15.196.212:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 15:52:30.463765 25688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:30.453131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:42097 (local address 127.15.196.254:33439)
0929 15:52:30.453328 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:30.453334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:30.454897 (+ 1563us) server_negotiation.cc:407] Connection header received
0929 15:52:30.456885 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:30.456890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:30.456975 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:30.457091 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:30.458177 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:30.458891 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:30.460603 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:30.461072 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:30.461180 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:30.462818 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:30.462828 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:30.462836 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:52:30.462862 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:30.462906 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:52:30.462925 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:30.463023 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:30.463235 (+ 212us) server_negotiation.cc:299] Negotiation successful
0929 15:52:30.463374 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 15:52:30.464742 25819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:30.453792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:33439 (local address 127.15.196.194:42097)
0929 15:52:30.454757 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:30.454792 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:30.456613 (+ 1821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:30.457317 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:30.457325 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:30.457419 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:30.458014 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:30.458031 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:30.459040 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:30.459045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:30.460437 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:30.460466 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:30.461314 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:30.461322 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:30.461683 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:30.462472 (+ 789us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:52:30.462474 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:30.462518 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:52:30.462604 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:30.463026 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:30.463033 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:30.463039 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 15:52:30.463193 (+ 154us) client_negotiation.cc:240] Negotiation successful
0929 15:52:30.463574 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":298,"threads_started":1}
I20260929 15:52:30.466307 25817 heartbeater.cc:347] Connected to a master server at 127.15.196.254:33439
I20260929 15:52:30.466650 25817 heartbeater.cc:464] Registering TS with master...
I20260929 15:52:30.467651 25817 heartbeater.cc:511] Master 127.15.196.254:33439 requested a full tablet report, sending...
I20260929 15:52:30.469280 25500 ts_manager.cc:194] Registered new tserver with Master: a57059a5b53d4ad89b5c5a0cb73b567a (127.15.196.194:44839)
I20260929 15:52:30.470515 25500 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.196.194:42097
W20260929 15:52:30.789659 25820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:30.789948 25820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:30.790041 25820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:30.797292 25820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:30.797408 25820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:30.797513 25820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:52:30.806517 25820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:33439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:30.807801 25820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:30.809675 25820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:30.822978 25828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:30.823203 25826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:30.823166 25825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:30.824683 25820 server_base.cc:1034] running on GCE node
I20260929 15:52:30.825383 25820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:30.826318 25820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:30.827533 25820 hybrid_clock.cc:648] HybridClock initialized: now 1790697150827483 us; error 61 us; skew 500 ppm
I20260929 15:52:30.827852 25820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:52:30.829970 25820 webserver.cc:460] Webserver started at http://127.15.196.195:37729/ using document root <none> and password file <none>
I20260929 15:52:30.830669 25820 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:30.830791 25820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:30.831110 25820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:30.834081 25820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "239c337dc0aa4cc28390ab662d017538"
format_stamp: "Formatted at 2026-09-29 15:52:30 on dist-test-slave-c51v"
server_key: "16137b0426b7163086a780240aac2deb"
server_key_iv: "878d7c11f103dd7d49fba91bd69c00ef"
server_key_version: "encryptionkey@0"
I20260929 15:52:30.834851 25820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "239c337dc0aa4cc28390ab662d017538"
format_stamp: "Formatted at 2026-09-29 15:52:30 on dist-test-slave-c51v"
server_key: "16137b0426b7163086a780240aac2deb"
server_key_iv: "878d7c11f103dd7d49fba91bd69c00ef"
server_key_version: "encryptionkey@0"
I20260929 15:52:30.839934 25820 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 15:52:30.843246 25834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:30.844652 25820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:52:30.844828 25820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "239c337dc0aa4cc28390ab662d017538"
format_stamp: "Formatted at 2026-09-29 15:52:30 on dist-test-slave-c51v"
server_key: "16137b0426b7163086a780240aac2deb"
server_key_iv: "878d7c11f103dd7d49fba91bd69c00ef"
server_key_version: "encryptionkey@0"
I20260929 15:52:30.845041 25820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:30.858342 25820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:30.859179 25820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:30.859494 25820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:30.860417 25820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:52:30.861871 25820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:30.861992 25820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:30.862131 25820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:30.862216 25820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:30.894678 25820 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:43559
I20260929 15:52:30.894767 25946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:43559 every 8 connection(s)
I20260929 15:52:30.896073 25820 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:52:30.899729 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 25820
I20260929 15:52:30.899900 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:52:30.900413 16147 external_mini_cluster.cc:1428] Setting key 3c39512e0c9d3c1aac8daa0e208607c1
I20260929 15:52:30.908394 25688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:30.898561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:54629 (local address 127.15.196.254:33439)
0929 15:52:30.898743 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:30.898748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:30.900261 (+ 1513us) server_negotiation.cc:407] Connection header received
0929 15:52:30.902139 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:30.902144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:30.902232 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:30.902339 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:30.903431 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:30.904156 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:30.905692 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:30.906110 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:30.906241 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:30.907572 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:30.907577 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:30.907581 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:52:30.907598 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:30.907632 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:52:30.907645 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:30.907736 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:30.907965 (+ 229us) server_negotiation.cc:299] Negotiation successful
0929 15:52:30.908056 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 15:52:30.909418 25949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:30.899084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:33439 (local address 127.15.196.195:54629)
0929 15:52:30.900106 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:30.900151 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:30.901872 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:30.902568 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:30.902576 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:30.902673 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:30.903226 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:30.903242 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:30.904312 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:30.904318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:30.905544 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:30.905555 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:30.906338 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:30.906345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:30.906582 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:30.907218 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:52:30.907220 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:30.907256 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:52:30.907385 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:30.907734 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:30.907746 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:30.907757 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 15:52:30.907935 (+ 178us) client_negotiation.cc:240] Negotiation successful
0929 15:52:30.908310 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":318,"threads_started":1}
I20260929 15:52:30.910797 25947 heartbeater.cc:347] Connected to a master server at 127.15.196.254:33439
I20260929 15:52:30.911098 25947 heartbeater.cc:464] Registering TS with master...
I20260929 15:52:30.911907 25947 heartbeater.cc:511] Master 127.15.196.254:33439 requested a full tablet report, sending...
I20260929 15:52:30.913275 25500 ts_manager.cc:194] Registered new tserver with Master: 239c337dc0aa4cc28390ab662d017538 (127.15.196.195:43559)
I20260929 15:52:30.914345 25500 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.196.195:54629
I20260929 15:52:30.915422 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 15:52:30.921569 25688 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:30.919964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53100 (local address 127.15.196.254:33439)
0929 15:52:30.920220 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:30.920230 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:30.920562 (+ 332us) server_negotiation.cc:407] Connection header received
0929 15:52:30.920707 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:30.920711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:30.920838 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:30.920937 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:30.921345 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53100: BlockingRecv error: recv got EOF from 127.0.0.1:53100 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 15:52:30.922077 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 25559
I20260929 15:52:30.929811 25956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:30.922997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55840 (local address 127.15.196.193:44739)
0929 15:52:30.923598 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:30.923606 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:30.923638 (+ 32us) server_negotiation.cc:407] Connection header received
0929 15:52:30.923708 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:30.923716 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:30.923951 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:30.924075 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:30.924410 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:30.925421 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:30.926658 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:30.927214 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:30.927366 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:30.928375 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:30.928383 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:30.928396 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:52:30.929017 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:30.929064 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:52:30.929083 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:30.929179 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:30.929302 (+ 123us) server_negotiation.cc:299] Negotiation successful
0929 15:52:30.929453 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":302,"threads_started":1}
I20260929 15:52:30.931057 25681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:31.014844 25686 heartbeater.cc:503] Master 127.15.196.254:33439 was elected leader, sending a full tablet report...
I20260929 15:52:31.016729 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 25559
W20260929 15:52:31.035483 25484 connection.cc:535] server connection from 127.15.196.193:36721 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 15:52:31.035769 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 25690
I20260929 15:52:31.043576 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:31.036638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43740 (local address 127.15.196.194:44839)
0929 15:52:31.037121 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:31.037130 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:31.037160 (+ 30us) server_negotiation.cc:407] Connection header received
0929 15:52:31.037229 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:31.037237 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:31.037458 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:31.037659 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:31.038047 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:31.039017 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:31.040421 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:31.040975 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:31.041118 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:31.042143 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:31.042158 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:31.042173 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:52:31.042766 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:31.042812 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:52:31.042833 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:31.042969 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:31.043084 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 15:52:31.043217 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":238,"threads_started":1}
I20260929 15:52:31.044919 25812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:31.128254 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 25690
I20260929 15:52:31.144170 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 25820
I20260929 15:52:31.152450 25960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:31.145234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44046 (local address 127.15.196.195:43559)
0929 15:52:31.145853 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:31.145860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:31.145888 (+ 28us) server_negotiation.cc:407] Connection header received
0929 15:52:31.145949 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:31.145956 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:31.146174 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:31.146307 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:31.146684 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:31.147863 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:31.149131 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:31.149689 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:31.149785 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:31.150952 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:31.150960 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:31.150975 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:52:31.151633 (+ 658us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:31.151686 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:52:31.151709 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:31.151794 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:31.151929 (+ 135us) server_negotiation.cc:299] Negotiation successful
0929 15:52:31.152068 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":310,"threads_started":1}
I20260929 15:52:31.153680 25942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:31.237579 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 25820
I20260929 15:52:31.255422 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 25469
I20260929 15:52:31.256832 25530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:31.362280 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 25469
2026-09-29T15:52:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2463 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-29T15:52:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:52:31Z Disabled control of system clock
I20260929 15:52:31.432247 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:38631
--webserver_interface=127.15.196.254
--webserver_port=0
--builtin_ntp_servers=127.15.196.212:46545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:38631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 15:52:31.755023 25966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:31.755347 25966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:31.755435 25966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:31.762600 25966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:52:31.762717 25966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:31.762754 25966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:31.762811 25966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:52:31.762856 25966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:52:31.772177 25966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:46545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:38631
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:38631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:31.773422 25966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:31.775146 25966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:31.786728 25971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:31.786757 25972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:31.786801 25974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:31.787482 25966 server_base.cc:1034] running on GCE node
I20260929 15:52:31.788237 25966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:31.789603 25966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:31.790825 25966 hybrid_clock.cc:648] HybridClock initialized: now 1790697151790787 us; error 53 us; skew 500 ppm
I20260929 15:52:31.791169 25966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:52:31.793378 25966 webserver.cc:460] Webserver started at http://127.15.196.254:37595/ using document root <none> and password file <none>
I20260929 15:52:31.794092 25966 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:31.794215 25966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:31.794521 25966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:31.797451 25966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "f73185d3755f408a99d9727d85569f7c"
format_stamp: "Formatted at 2026-09-29 15:52:31 on dist-test-slave-c51v"
server_key: "d6871236401afefadc8f4cee7c84d7a8"
server_key_iv: "b49bdad6759547d6bd2a0263d8feccc0"
server_key_version: "encryptionkey@0"
I20260929 15:52:31.798264 25966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "f73185d3755f408a99d9727d85569f7c"
format_stamp: "Formatted at 2026-09-29 15:52:31 on dist-test-slave-c51v"
server_key: "d6871236401afefadc8f4cee7c84d7a8"
server_key_iv: "b49bdad6759547d6bd2a0263d8feccc0"
server_key_version: "encryptionkey@0"
I20260929 15:52:31.803257 25966 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 15:52:31.806782 25980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:31.808211 25966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:52:31.808387 25966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "f73185d3755f408a99d9727d85569f7c"
format_stamp: "Formatted at 2026-09-29 15:52:31 on dist-test-slave-c51v"
server_key: "d6871236401afefadc8f4cee7c84d7a8"
server_key_iv: "b49bdad6759547d6bd2a0263d8feccc0"
server_key_version: "encryptionkey@0"
I20260929 15:52:31.808624 25966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:31.814229 25966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:31.815004 25966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:31.815392 25966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:31.833254 25966 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:38631
I20260929 15:52:31.833287 26031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:38631 every 8 connection(s)
I20260929 15:52:31.834522 25966 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:52:31.838178 26032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:31.841812 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 25966
I20260929 15:52:31.842046 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:52:31.842859 16147 external_mini_cluster.cc:1428] Setting key fcad381c6a30d4d0f6a566c456aefd82
I20260929 15:52:31.851076 26035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:31.844974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60348 (local address 127.15.196.254:38631)
0929 15:52:31.846000 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:31.846011 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:31.846053 (+ 42us) server_negotiation.cc:407] Connection header received
0929 15:52:31.847668 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:31.847691 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:31.848123 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:31.848456 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:31.849530 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:31.849537 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:31.849548 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:52:31.849561 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:31.849620 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:52:31.849635 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:31.849830 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:31.849933 (+ 103us) server_negotiation.cc:299] Negotiation successful
0929 15:52:31.850218 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":386,"threads_started":1}
I20260929 15:52:31.851742 26032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c: Bootstrap starting.
I20260929 15:52:31.854949 26032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:31.855952 26032 log.cc:824] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c: Log is configured to *not* fsync() on all Append() calls
I20260929 15:52:31.858585 26032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c: No bootstrap required, opened a new log
I20260929 15:52:31.865935 26032 raft_consensus.cc:348] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73185d3755f408a99d9727d85569f7c" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38631 } }
I20260929 15:52:31.866288 26032 raft_consensus.cc:374] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:31.866351 26032 raft_consensus.cc:729] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f73185d3755f408a99d9727d85569f7c, State: Initialized, Role: FOLLOWER
I20260929 15:52:31.866839 26032 consensus_queue.cc:260] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73185d3755f408a99d9727d85569f7c" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38631 } }
I20260929 15:52:31.867027 26032 raft_consensus.cc:388] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:52:31.867130 26032 raft_consensus.cc:482] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:52:31.867250 26032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:31.869596 26032 raft_consensus.cc:504] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73185d3755f408a99d9727d85569f7c" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38631 } }
I20260929 15:52:31.870204 26032 leader_election.cc:302] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f73185d3755f408a99d9727d85569f7c; no voters:
I20260929 15:52:31.870913 26032 leader_election.cc:288] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:52:31.870992 26037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:52:31.871845 26037 raft_consensus.cc:686] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [term 1 LEADER]: Becoming Leader. State: Replica: f73185d3755f408a99d9727d85569f7c, State: Running, Role: LEADER
I20260929 15:52:31.872399 26037 consensus_queue.cc:237] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73185d3755f408a99d9727d85569f7c" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38631 } }
I20260929 15:52:31.873235 26032 sys_catalog.cc:564] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:52:31.874984 26038 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f73185d3755f408a99d9727d85569f7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73185d3755f408a99d9727d85569f7c" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38631 } } }
I20260929 15:52:31.875025 26039 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f73185d3755f408a99d9727d85569f7c. Latest consensus state: current_term: 1 leader_uuid: "f73185d3755f408a99d9727d85569f7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73185d3755f408a99d9727d85569f7c" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38631 } } }
I20260929 15:52:31.875439 26039 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [sys.catalog]: This master's current role is: LEADER
I20260929 15:52:31.875437 26038 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c [sys.catalog]: This master's current role is: LEADER
I20260929 15:52:31.876420 26042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:52:31.881567 26042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:52:31.891649 26042 catalog_manager.cc:1347] Generated new cluster ID: c308da4205af4ed58b210ebe28b8ee54
I20260929 15:52:31.891772 26042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:52:31.915578 26042 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:52:31.916011 26042 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:52:31.924953 26042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f73185d3755f408a99d9727d85569f7c: Generated new TSK 0
I20260929 15:52:31.925823 26042 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:52:31.933511 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:38631
--builtin_ntp_servers=127.15.196.212:46545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 15:52:32.324993 26056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:32.325353 26056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:32.325528 26056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:32.333216 26056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:32.333312 26056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:32.333453 26056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:52:32.343215 26056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:46545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:38631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:32.344929 26056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:32.346726 26056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:32.359613 26061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:32.359644 26062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:32.359674 26064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:32.360420 26056 server_base.cc:1034] running on GCE node
I20260929 15:52:32.360973 26056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:32.361850 26056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:32.363040 26056 hybrid_clock.cc:648] HybridClock initialized: now 1790697152362961 us; error 72 us; skew 500 ppm
I20260929 15:52:32.363407 26056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:52:32.365774 26056 webserver.cc:460] Webserver started at http://127.15.196.193:42689/ using document root <none> and password file <none>
I20260929 15:52:32.366494 26056 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:32.366607 26056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:32.366940 26056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:32.369871 26056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "df60f293871843029c4df1f9474ebcfa"
format_stamp: "Formatted at 2026-09-29 15:52:32 on dist-test-slave-c51v"
server_key: "6309314b61b96334b23c2fdf3f4af98a"
server_key_iv: "288d0448f4995b2f33d764e86d605c21"
server_key_version: "encryptionkey@0"
I20260929 15:52:32.370676 26056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "df60f293871843029c4df1f9474ebcfa"
format_stamp: "Formatted at 2026-09-29 15:52:32 on dist-test-slave-c51v"
server_key: "6309314b61b96334b23c2fdf3f4af98a"
server_key_iv: "288d0448f4995b2f33d764e86d605c21"
server_key_version: "encryptionkey@0"
I20260929 15:52:32.376165 26056 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 15:52:32.379601 26070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:32.380966 26056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:52:32.381129 26056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "df60f293871843029c4df1f9474ebcfa"
format_stamp: "Formatted at 2026-09-29 15:52:32 on dist-test-slave-c51v"
server_key: "6309314b61b96334b23c2fdf3f4af98a"
server_key_iv: "288d0448f4995b2f33d764e86d605c21"
server_key_version: "encryptionkey@0"
I20260929 15:52:32.381356 26056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:32.387019 26056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:32.387833 26056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:32.388057 26056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:32.388876 26056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:52:32.390229 26056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:32.390312 26056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:32.390390 26056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:32.390444 26056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:32.422487 26056 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:45469
I20260929 15:52:32.422537 26182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:45469 every 8 connection(s)
I20260929 15:52:32.423914 26056 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:52:32.430473 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 26056
I20260929 15:52:32.430701 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:52:32.431366 16147 external_mini_cluster.cc:1428] Setting key 49231b614b93491e981605f51560d3a0
I20260929 15:52:32.432520 26185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:32.426522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:55021 (local address 127.15.196.254:38631)
0929 15:52:32.426959 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:32.426963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:32.428314 (+ 1351us) server_negotiation.cc:407] Connection header received
0929 15:52:32.430027 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:32.430034 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:32.430108 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:32.430224 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:32.431718 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:32.431724 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:32.431729 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:52:32.431748 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:32.431800 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:52:32.431812 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:32.431945 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:32.432130 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 15:52:32.432202 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":236,"threads_started":1}
I20260929 15:52:32.433475 26186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:32.427097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38631 (local address 127.15.196.193:55021)
0929 15:52:32.428152 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:32.428196 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:32.429733 (+ 1537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:32.430454 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:32.430462 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:32.430556 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:32.431356 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:52:32.431359 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:32.431407 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:52:32.431535 (+ 128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:32.431905 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:32.431916 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:32.431926 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 15:52:32.432225 (+ 299us) client_negotiation.cc:240] Negotiation successful
0929 15:52:32.432498 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":375,"threads_started":1}
I20260929 15:52:32.434355 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:38631
--builtin_ntp_servers=127.15.196.212:46545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260929 15:52:32.435042 26183 heartbeater.cc:347] Connected to a master server at 127.15.196.254:38631
I20260929 15:52:32.435432 26183 heartbeater.cc:464] Registering TS with master...
I20260929 15:52:32.436020 26183 heartbeater.cc:511] Master 127.15.196.254:38631 requested a full tablet report, sending...
I20260929 15:52:32.438246 25997 ts_manager.cc:194] Registered new tserver with Master: df60f293871843029c4df1f9474ebcfa (127.15.196.193:45469)
W20260929 15:52:32.763495 26187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:32.763788 26187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:32.763880 26187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:32.771579 26187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:32.771698 26187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:32.771807 26187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:52:32.781414 26187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:46545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:38631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:32.782866 26187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:32.784816 26187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:32.798638 26192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:32.799229 26195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:32.799255 26193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:32.799414 26187 server_base.cc:1034] running on GCE node
I20260929 15:52:32.800108 26187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:32.801070 26187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:32.802281 26187 hybrid_clock.cc:648] HybridClock initialized: now 1790697152802241 us; error 39 us; skew 500 ppm
I20260929 15:52:32.802598 26187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:52:32.804827 26187 webserver.cc:460] Webserver started at http://127.15.196.194:39807/ using document root <none> and password file <none>
I20260929 15:52:32.805552 26187 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:32.805692 26187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:32.806013 26187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:32.810793 26187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "710104fb4d324d3fb80f67a980511d8d"
format_stamp: "Formatted at 2026-09-29 15:52:32 on dist-test-slave-c51v"
server_key: "ab94efc397bd18d3ae992828800bd70c"
server_key_iv: "057c951377922c5b86538f8424ba6578"
server_key_version: "encryptionkey@0"
I20260929 15:52:32.811765 26187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "710104fb4d324d3fb80f67a980511d8d"
format_stamp: "Formatted at 2026-09-29 15:52:32 on dist-test-slave-c51v"
server_key: "ab94efc397bd18d3ae992828800bd70c"
server_key_iv: "057c951377922c5b86538f8424ba6578"
server_key_version: "encryptionkey@0"
I20260929 15:52:32.816883 26187 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 15:52:32.820369 26201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:32.821673 26187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 15:52:32.821843 26187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "710104fb4d324d3fb80f67a980511d8d"
format_stamp: "Formatted at 2026-09-29 15:52:32 on dist-test-slave-c51v"
server_key: "ab94efc397bd18d3ae992828800bd70c"
server_key_iv: "057c951377922c5b86538f8424ba6578"
server_key_version: "encryptionkey@0"
I20260929 15:52:32.822058 26187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:32.827553 26187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:32.828325 26187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:32.828591 26187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:32.829500 26187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:52:32.830888 26187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:32.831007 26187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:32.831135 26187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:32.831220 26187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:32.863855 26187 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:45831
I20260929 15:52:32.863915 26313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:45831 every 8 connection(s)
I20260929 15:52:32.865228 26187 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:52:32.866662 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 26187
I20260929 15:52:32.866864 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:52:32.867358 16147 external_mini_cluster.cc:1428] Setting key 81bec5e9bd9732f984b30202aa21fd26
I20260929 15:52:32.870685 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:38631
--builtin_ntp_servers=127.15.196.212:46545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260929 15:52:32.875092 26185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:32.867919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:58533 (local address 127.15.196.254:38631)
0929 15:52:32.868110 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:32.868118 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:32.869941 (+ 1823us) server_negotiation.cc:407] Connection header received
0929 15:52:32.872743 (+ 2802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:32.872748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:32.872802 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:32.872895 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:32.874396 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:32.874401 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:32.874405 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:52:32.874422 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:32.874456 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:52:32.874485 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:32.874599 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:32.874825 (+ 226us) server_negotiation.cc:299] Negotiation successful
0929 15:52:32.874881 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 15:52:32.876456 26316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:32.868596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38631 (local address 127.15.196.194:58533)
0929 15:52:32.869771 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:32.869831 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:32.872435 (+ 2604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:32.873157 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:32.873172 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:32.873302 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:32.874026 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:52:32.874032 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:32.874087 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:52:32.874203 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:32.874586 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:32.874599 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:32.874610 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 15:52:32.874888 (+ 278us) client_negotiation.cc:240] Negotiation successful
0929 15:52:32.875160 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":332,"threads_started":1}
I20260929 15:52:32.877998 26314 heartbeater.cc:347] Connected to a master server at 127.15.196.254:38631
I20260929 15:52:32.878336 26314 heartbeater.cc:464] Registering TS with master...
I20260929 15:52:32.879405 26314 heartbeater.cc:511] Master 127.15.196.254:38631 requested a full tablet report, sending...
I20260929 15:52:32.880934 25997 ts_manager.cc:194] Registered new tserver with Master: 710104fb4d324d3fb80f67a980511d8d (127.15.196.194:45831)
W20260929 15:52:33.210861 26317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:33.211189 26317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:33.211393 26317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:33.218767 26317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:33.218914 26317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:33.219097 26317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:52:33.228538 26317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:46545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:38631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:33.229875 26317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:33.231789 26317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:33.245079 26325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:33.245178 26323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:33.245191 26322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:33.248734 26317 server_base.cc:1034] running on GCE node
I20260929 15:52:33.249336 26317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:33.250283 26317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:33.251503 26317 hybrid_clock.cc:648] HybridClock initialized: now 1790697153251462 us; error 59 us; skew 500 ppm
I20260929 15:52:33.251827 26317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:52:33.254166 26317 webserver.cc:460] Webserver started at http://127.15.196.195:42405/ using document root <none> and password file <none>
I20260929 15:52:33.254904 26317 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:33.255026 26317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:33.255391 26317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:33.258275 26317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "d0b0b0c4f4404fa3b2bada1a5cc01004"
format_stamp: "Formatted at 2026-09-29 15:52:33 on dist-test-slave-c51v"
server_key: "8da2ac1e81c94987faae776943cad924"
server_key_iv: "ecad2169f438828320cb83761497b3f6"
server_key_version: "encryptionkey@0"
I20260929 15:52:33.259080 26317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "d0b0b0c4f4404fa3b2bada1a5cc01004"
format_stamp: "Formatted at 2026-09-29 15:52:33 on dist-test-slave-c51v"
server_key: "8da2ac1e81c94987faae776943cad924"
server_key_iv: "ecad2169f438828320cb83761497b3f6"
server_key_version: "encryptionkey@0"
I20260929 15:52:33.264277 26317 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 15:52:33.267635 26331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:33.268980 26317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260929 15:52:33.269160 26317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "d0b0b0c4f4404fa3b2bada1a5cc01004"
format_stamp: "Formatted at 2026-09-29 15:52:33 on dist-test-slave-c51v"
server_key: "8da2ac1e81c94987faae776943cad924"
server_key_iv: "ecad2169f438828320cb83761497b3f6"
server_key_version: "encryptionkey@0"
I20260929 15:52:33.269382 26317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:33.275350 26317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:33.276121 26317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:33.276382 26317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:33.277242 26317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:52:33.278621 26317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:33.278744 26317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:33.278875 26317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:33.278959 26317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:33.310856 26317 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:46433
I20260929 15:52:33.310911 26443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:46433 every 8 connection(s)
I20260929 15:52:33.312206 26317 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:52:33.316035 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 26317
I20260929 15:52:33.316262 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:52:33.316835 16147 external_mini_cluster.cc:1428] Setting key a7888634abe363add0845d4369e0f30e
I20260929 15:52:33.321300 26185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:33.314785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:36017 (local address 127.15.196.254:38631)
0929 15:52:33.314955 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:33.314960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:33.316825 (+ 1865us) server_negotiation.cc:407] Connection header received
0929 15:52:33.318933 (+ 2108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:33.318947 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:33.319023 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:33.319129 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:33.320571 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:33.320577 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:33.320582 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:52:33.320600 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:33.320638 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:52:33.320655 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:33.320770 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:33.320949 (+ 179us) server_negotiation.cc:299] Negotiation successful
0929 15:52:33.321014 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 15:52:33.322191 26446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:33.315409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38631 (local address 127.15.196.195:36017)
0929 15:52:33.316655 (+ 1246us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:33.316705 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:33.318681 (+ 1976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:33.319404 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:33.319417 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:33.319510 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:33.320293 (+ 783us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:52:33.320295 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:33.320333 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:52:33.320414 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:33.320744 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:33.320754 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:33.320762 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 15:52:33.321040 (+ 278us) client_negotiation.cc:240] Negotiation successful
0929 15:52:33.321288 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":351,"threads_started":1}
I20260929 15:52:33.323417 26444 heartbeater.cc:347] Connected to a master server at 127.15.196.254:38631
I20260929 15:52:33.323763 26444 heartbeater.cc:464] Registering TS with master...
I20260929 15:52:33.324362 26444 heartbeater.cc:511] Master 127.15.196.254:38631 requested a full tablet report, sending...
I20260929 15:52:33.325690 25997 ts_manager.cc:194] Registered new tserver with Master: d0b0b0c4f4404fa3b2bada1a5cc01004 (127.15.196.195:46433)
I20260929 15:52:33.331911 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 15:52:33.337780 26185 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:52:33.336359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60354 (local address 127.15.196.254:38631)
0929 15:52:33.336514 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:33.336518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:33.337011 (+ 493us) server_negotiation.cc:407] Connection header received
0929 15:52:33.337151 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:33.337155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:33.337205 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:33.337286 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:33.337568 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60354: BlockingRecv error: recv got EOF from 127.0.0.1:60354 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 15:52:33.338238 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 26056
I20260929 15:52:33.341735 26453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:33.339123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47538 (local address 127.15.196.193:45469)
0929 15:52:33.339634 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:33.339640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:33.339666 (+ 26us) server_negotiation.cc:407] Connection header received
0929 15:52:33.339719 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:33.339725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:33.339925 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:33.340080 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:33.341063 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:33.341070 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:33.341081 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:52:33.341097 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:33.341135 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:52:33.341148 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:33.341294 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:33.341373 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 15:52:33.341470 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":230,"threads_started":1}
I20260929 15:52:33.342882 26178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:33.427438 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 26056
I20260929 15:52:33.444649 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 26187
I20260929 15:52:33.448253 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:33.445663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37076 (local address 127.15.196.194:45831)
0929 15:52:33.446160 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:33.446167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:33.446195 (+ 28us) server_negotiation.cc:407] Connection header received
0929 15:52:33.446258 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:33.446264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:33.446502 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:33.446633 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:33.447624 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:33.447632 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:33.447645 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:52:33.447679 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:33.447737 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:52:33.447753 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:33.447863 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:33.447941 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 15:52:33.448026 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":226,"threads_started":1}
I20260929 15:52:33.449491 26309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:33.535392 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 26187
I20260929 15:52:33.552452 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 26317
I20260929 15:52:33.556160 26457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:33.553522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33160 (local address 127.15.196.195:46433)
0929 15:52:33.554117 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:33.554124 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:33.554153 (+ 29us) server_negotiation.cc:407] Connection header received
0929 15:52:33.554215 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:33.554221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:33.554438 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:33.554561 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:33.555530 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:33.555536 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:33.555548 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:52:33.555561 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:33.555601 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:52:33.555615 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:33.555758 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:33.555810 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 15:52:33.555889 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":298,"threads_started":1}
I20260929 15:52:33.557436 26439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:33.644088 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 26317
I20260929 15:52:33.660203 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 25966
I20260929 15:52:33.661367 26027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:33.764741 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 25966
2026-09-29T15:52:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2402 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:52:33 dist-test-slave-c51v krb5kdc[26462](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:52:33 dist-test-slave-c51v krb5kdc[26462](info): set up 2 sockets
Sep 29 15:52:33 dist-test-slave-c51v krb5kdc[26462](info): commencing operation
krb5kdc: starting...
W20260929 15:52:34.918628 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:52:35 dist-test-slave-c51v krb5kdc[26462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:52:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:52:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:52:35.159945 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:35139
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:42579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:35139
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:52:35.500445 26475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:35.500818 26475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:35.501006 26475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:35.509436 26475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:52:35.509569 26475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:35.509624 26475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:35.509692 26475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:52:35.509773 26475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:52:35.520406 26475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:42579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:35139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:35139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:35.521889 26475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:35.523435 26475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:35.536374 26480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:35.536343 26483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:35.536459 26481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:35.537186 26475 server_base.cc:1034] running on GCE node
I20260929 15:52:35.538064 26475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:35.539695 26475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:35.540910 26475 hybrid_clock.cc:648] HybridClock initialized: now 1790697155540860 us; error 36 us; skew 500 ppm
I20260929 15:52:35.541285 26475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:52:35 dist-test-slave-c51v krb5kdc[26462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697155, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:52:35.548151 26475 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:52:35.549806 26475 webserver.cc:460] Webserver started at http://127.15.196.254:41885/ using document root <none> and password file <none>
I20260929 15:52:35.550549 26475 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:35.550712 26475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:35.551034 26475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:35.554196 26475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "a2e749b75b07403fb5daadb937ea2d63"
format_stamp: "Formatted at 2026-09-29 15:52:35 on dist-test-slave-c51v"
server_key: "078e0f45f5ee593ad6a22f00a51bf9ce"
server_key_iv: "b3925931c1335032010fa2ca87528998"
server_key_version: "encryptionkey@0"
I20260929 15:52:35.554976 26475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "a2e749b75b07403fb5daadb937ea2d63"
format_stamp: "Formatted at 2026-09-29 15:52:35 on dist-test-slave-c51v"
server_key: "078e0f45f5ee593ad6a22f00a51bf9ce"
server_key_iv: "b3925931c1335032010fa2ca87528998"
server_key_version: "encryptionkey@0"
I20260929 15:52:35.560508 26475 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260929 15:52:35.564173 26490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:35.565593 26475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:52:35.565794 26475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "a2e749b75b07403fb5daadb937ea2d63"
format_stamp: "Formatted at 2026-09-29 15:52:35 on dist-test-slave-c51v"
server_key: "078e0f45f5ee593ad6a22f00a51bf9ce"
server_key_iv: "b3925931c1335032010fa2ca87528998"
server_key_version: "encryptionkey@0"
I20260929 15:52:35.566006 26475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:35.578550 26475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:35.582818 26475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:35.583070 26475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:35.601702 26475 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:35139
I20260929 15:52:35.601719 26541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:35139 every 8 connection(s)
I20260929 15:52:35.603137 26475 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:52:35.607110 26542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:35.611918 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 26475
I20260929 15:52:35.612174 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:52:35.613003 16147 external_mini_cluster.cc:1428] Setting key 2da4256fdfc47310fc88052a8f31d3e4
I20260929 15:52:35.618916 26542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63: Bootstrap starting.
I20260929 15:52:35.622097 26542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:35.623106 26542 log.cc:824] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63: Log is configured to *not* fsync() on all Append() calls
I20260929 15:52:35.626000 26542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63: No bootstrap required, opened a new log
Sep 29 15:52:35 dist-test-slave-c51v krb5kdc[26462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:52:35.635217 26542 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e749b75b07403fb5daadb937ea2d63" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35139 } }
I20260929 15:52:35.635649 26542 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:35.635782 26542 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2e749b75b07403fb5daadb937ea2d63, State: Initialized, Role: FOLLOWER
I20260929 15:52:35.635978 26545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:35.615164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.15.196.254:35139)
0929 15:52:35.616004 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:35.616020 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:35.616062 (+ 42us) server_negotiation.cc:407] Connection header received
0929 15:52:35.617784 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:35.617809 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:35.618225 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:35.618625 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:35.620168 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:35.621474 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:35.622823 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:35.623680 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:35.623888 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:35.628270 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:35.628278 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:35.628295 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:35.628330 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:35.631765 (+ 3435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:35.632631 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:35.632638 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:35.632646 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:35.632803 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:35.633212 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:35.633215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:35.633218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:35.633742 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:35.633965 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:35.634402 (+ 437us) server_negotiation.cc:299] Negotiation successful
0929 15:52:35.634770 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":367,"threads_started":1}
I20260929 15:52:35.636494 26542 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e749b75b07403fb5daadb937ea2d63" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35139 } }
I20260929 15:52:35.636761 26542 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:52:35.636902 26542 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:52:35.637092 26542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:35.639572 26542 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e749b75b07403fb5daadb937ea2d63" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35139 } }
I20260929 15:52:35.640041 26542 leader_election.cc:302] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2e749b75b07403fb5daadb937ea2d63; no voters:
I20260929 15:52:35.640627 26542 leader_election.cc:288] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:52:35.640756 26547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:52:35.641480 26547 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [term 1 LEADER]: Becoming Leader. State: Replica: a2e749b75b07403fb5daadb937ea2d63, State: Running, Role: LEADER
I20260929 15:52:35.641991 26547 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e749b75b07403fb5daadb937ea2d63" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35139 } }
I20260929 15:52:35.642684 26542 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:52:35.645516 26548 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2e749b75b07403fb5daadb937ea2d63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e749b75b07403fb5daadb937ea2d63" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35139 } } }
I20260929 15:52:35.645814 26549 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2e749b75b07403fb5daadb937ea2d63. Latest consensus state: current_term: 1 leader_uuid: "a2e749b75b07403fb5daadb937ea2d63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e749b75b07403fb5daadb937ea2d63" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35139 } } }
I20260929 15:52:35.646130 26548 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [sys.catalog]: This master's current role is: LEADER
I20260929 15:52:35.646163 26549 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63 [sys.catalog]: This master's current role is: LEADER
I20260929 15:52:35.647876 26553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:52:35.653805 26553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:52:35.662058 26553 catalog_manager.cc:1347] Generated new cluster ID: 20cacdc8b10040d7998d66167b8692ad
I20260929 15:52:35.662191 26553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:52:35.711221 26553 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:52:35.712656 26553 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:52:35.726339 26553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63: Generated new TSK 0
I20260929 15:52:35.727316 26553 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:52:35.787873 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:35139
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:42579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:52:36.125959 26568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:36.126271 26568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:36.126443 26568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:36.134414 26568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:36.134534 26568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:36.134719 26568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:52:36.145202 26568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:42579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:35139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:36.146569 26568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:36.148175 26568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:36.161612 26573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:36.161688 26574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:36.161648 26576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:36.162640 26568 server_base.cc:1034] running on GCE node
I20260929 15:52:36.163331 26568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:36.164323 26568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:36.165556 26568 hybrid_clock.cc:648] HybridClock initialized: now 1790697156165482 us; error 64 us; skew 500 ppm
I20260929 15:52:36.165925 26568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:52:36 dist-test-slave-c51v krb5kdc[26462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697156, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:52:36.169222 26568 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:52:36.170742 26568 webserver.cc:460] Webserver started at http://127.15.196.193:34129/ using document root <none> and password file <none>
I20260929 15:52:36.171535 26568 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:36.171636 26568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:36.171900 26568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:36.174746 26568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "7b65b0f8fe7e4a7292ed597d67edce6a"
format_stamp: "Formatted at 2026-09-29 15:52:36 on dist-test-slave-c51v"
server_key: "172cb9976d60e3f2b4be81821109232d"
server_key_iv: "4b0c792979954b8d0e8ae4f02bb0e3f5"
server_key_version: "encryptionkey@0"
I20260929 15:52:36.175565 26568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "7b65b0f8fe7e4a7292ed597d67edce6a"
format_stamp: "Formatted at 2026-09-29 15:52:36 on dist-test-slave-c51v"
server_key: "172cb9976d60e3f2b4be81821109232d"
server_key_iv: "4b0c792979954b8d0e8ae4f02bb0e3f5"
server_key_version: "encryptionkey@0"
I20260929 15:52:36.180792 26568 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260929 15:52:36.184360 26583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:36.185814 26568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:52:36.186004 26568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "7b65b0f8fe7e4a7292ed597d67edce6a"
format_stamp: "Formatted at 2026-09-29 15:52:36 on dist-test-slave-c51v"
server_key: "172cb9976d60e3f2b4be81821109232d"
server_key_iv: "4b0c792979954b8d0e8ae4f02bb0e3f5"
server_key_version: "encryptionkey@0"
I20260929 15:52:36.186213 26568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:36.255190 26568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:36.259691 26568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:36.260007 26568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:36.261055 26568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:52:36.262514 26568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:36.262622 26568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:36.262784 26568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:36.262859 26568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:36.297828 26568 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:45103
I20260929 15:52:36.297873 26695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:45103 every 8 connection(s)
I20260929 15:52:36.299211 26568 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:52:36.302492 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 26568
I20260929 15:52:36.302712 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:52:36.303402 16147 external_mini_cluster.cc:1428] Setting key 3d0693bd474ac9d89e94aba83b230907
Sep 29 15:52:36 dist-test-slave-c51v krb5kdc[26462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697156, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:52:36.320217 26698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:36.301916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:49739 (local address 127.15.196.254:35139)
0929 15:52:36.302440 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:36.302447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:36.303710 (+ 1263us) server_negotiation.cc:407] Connection header received
0929 15:52:36.305520 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:36.305525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:36.305608 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:36.305735 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:36.307268 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:36.308025 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:36.309604 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:36.310026 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:36.310166 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:36.314083 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:36.314088 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:36.314091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:36.314124 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:36.316806 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:36.317803 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:36.317807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:36.317809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:36.317920 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:36.318391 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:36.318394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:36.318396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:36.318648 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:36.318757 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:36.319781 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0929 15:52:36.319956 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":255,"threads_started":1}
I20260929 15:52:36.321841 26699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:36.302575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:35139 (local address 127.15.196.193:49739)
0929 15:52:36.303529 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:36.303577 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:36.305249 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:36.305989 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:36.305997 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:36.306512 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:36.307101 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:36.307118 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:36.308191 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:36.308196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:36.309444 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:36.309456 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:36.310260 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:36.310268 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:36.310508 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:36.311206 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:36.311230 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:36.313848 (+ 2618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:36.316963 (+ 3115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:36.316984 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:36.317006 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:36.317673 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:36.318031 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:36.318039 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:36.318045 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:36.318284 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:36.318785 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:36.318796 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:36.319335 (+ 539us) client_negotiation.cc:769] Sending connection context
0929 15:52:36.319710 (+ 375us) client_negotiation.cc:240] Negotiation successful
0929 15:52:36.320311 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":443,"threads_started":1}
I20260929 15:52:36.323539 26696 heartbeater.cc:347] Connected to a master server at 127.15.196.254:35139
I20260929 15:52:36.323911 26696 heartbeater.cc:464] Registering TS with master...
I20260929 15:52:36.324935 26696 heartbeater.cc:511] Master 127.15.196.254:35139 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
I20260929 15:52:36.327231 26507 ts_manager.cc:194] Registered new tserver with Master: 7b65b0f8fe7e4a7292ed597d67edce6a (127.15.196.193:45103)
I20260929 15:52:36.328972 26507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:49739
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:52:36.358384 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:35139
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:42579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:52:36.682888 26702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:36.683172 26702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:36.683275 26702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:36.690922 26702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:36.691043 26702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:36.691207 26702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:52:36.701110 26702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:42579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:35139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:36.702402 26702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:36.703743 26702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:36.716817 26707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:36.716750 26710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:36.716846 26708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:36.717609 26702 server_base.cc:1034] running on GCE node
I20260929 15:52:36.718160 26702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:36.719066 26702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:36.720301 26702 hybrid_clock.cc:648] HybridClock initialized: now 1790697156720232 us; error 61 us; skew 500 ppm
I20260929 15:52:36.720623 26702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:52:36 dist-test-slave-c51v krb5kdc[26462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697156, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:52:36.723677 26702 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:52:36.725131 26702 webserver.cc:460] Webserver started at http://127.15.196.194:33627/ using document root <none> and password file <none>
I20260929 15:52:36.725809 26702 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:36.725946 26702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:36.726251 26702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:36.729306 26702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "61daba3d1f4047a0b6fe96d7827df058"
format_stamp: "Formatted at 2026-09-29 15:52:36 on dist-test-slave-c51v"
server_key: "a98e9260730d0cb2bf9066252b18f03e"
server_key_iv: "3803dd4357e7acc0f73f4dbed4e9fc35"
server_key_version: "encryptionkey@0"
I20260929 15:52:36.730135 26702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "61daba3d1f4047a0b6fe96d7827df058"
format_stamp: "Formatted at 2026-09-29 15:52:36 on dist-test-slave-c51v"
server_key: "a98e9260730d0cb2bf9066252b18f03e"
server_key_iv: "3803dd4357e7acc0f73f4dbed4e9fc35"
server_key_version: "encryptionkey@0"
I20260929 15:52:36.735271 26702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 15:52:36.738634 26717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:36.740082 26702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 15:52:36.740255 26702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "61daba3d1f4047a0b6fe96d7827df058"
format_stamp: "Formatted at 2026-09-29 15:52:36 on dist-test-slave-c51v"
server_key: "a98e9260730d0cb2bf9066252b18f03e"
server_key_iv: "3803dd4357e7acc0f73f4dbed4e9fc35"
server_key_version: "encryptionkey@0"
I20260929 15:52:36.740446 26702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:36.759516 26702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:36.763510 26702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:36.763800 26702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:36.764747 26702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:52:36.766098 26702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:36.766198 26702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:36.766350 26702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:36.766419 26702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:36.799666 26702 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:36491
I20260929 15:52:36.799701 26829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:36491 every 8 connection(s)
I20260929 15:52:36.801142 26702 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:52:36.811100 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 26702
I20260929 15:52:36.811367 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:52:36.811959 16147 external_mini_cluster.cc:1428] Setting key 83a4b84a5927269895ba4c0f0132da14
Sep 29 15:52:36 dist-test-slave-c51v krb5kdc[26462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697156, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:52:36.822165 26698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:36.803796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:39831 (local address 127.15.196.254:35139)
0929 15:52:36.803977 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:36.803981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:36.805154 (+ 1173us) server_negotiation.cc:407] Connection header received
0929 15:52:36.806886 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:36.806891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:36.806961 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:36.807051 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:36.808538 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:36.809247 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:36.810720 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:36.811116 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:36.811239 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:36.815116 (+ 3877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:36.815125 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:36.815133 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:36.815177 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:36.818881 (+ 3704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:36.819650 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:36.819659 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:36.819665 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:36.819823 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:36.820232 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:36.820240 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:36.820246 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:36.820635 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:36.820799 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:36.821457 (+ 658us) server_negotiation.cc:299] Negotiation successful
0929 15:52:36.821721 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 15:52:36.823014 26832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:36.804288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:35139 (local address 127.15.196.194:39831)
0929 15:52:36.804986 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:36.805025 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:36.806629 (+ 1604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:36.807278 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:36.807310 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:36.807821 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:36.808387 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:36.808405 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:36.809378 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:36.809381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:36.810602 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:36.810613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:36.811407 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:36.811417 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:36.811708 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:36.812416 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:36.812453 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:36.814887 (+ 2434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:36.819031 (+ 4144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:36.819039 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:36.819053 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:36.819531 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:36.819932 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:36.819935 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:36.819937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:36.820113 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:36.820765 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:36.820771 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:36.821084 (+ 313us) client_negotiation.cc:769] Sending connection context
0929 15:52:36.821313 (+ 229us) client_negotiation.cc:240] Negotiation successful
0929 15:52:36.821699 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":261,"threads_started":1}
I20260929 15:52:36.824633 26830 heartbeater.cc:347] Connected to a master server at 127.15.196.254:35139
I20260929 15:52:36.824961 26830 heartbeater.cc:464] Registering TS with master...
I20260929 15:52:36.825688 26830 heartbeater.cc:511] Master 127.15.196.254:35139 requested a full tablet report, sending...
I20260929 15:52:36.827155 26507 ts_manager.cc:194] Registered new tserver with Master: 61daba3d1f4047a0b6fe96d7827df058 (127.15.196.194:36491)
I20260929 15:52:36.828415 26507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:39831
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:52:36.863001 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:35139
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:42579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:52:37.188165 26835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:52:37.188459 26835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:52:37.188562 26835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:52:37.196426 26835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:52:37.196554 26835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:52:37.196676 26835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:52:37.206615 26835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:42579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:35139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:52:37.207947 26835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:52:37.209249 26835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:52:37.222637 26841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:37.222680 26840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:52:37.222723 26843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:52:37.223661 26835 server_base.cc:1034] running on GCE node
I20260929 15:52:37.224308 26835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:52:37.225303 26835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:52:37.226563 26835 hybrid_clock.cc:648] HybridClock initialized: now 1790697157226523 us; error 51 us; skew 500 ppm
I20260929 15:52:37.226936 26835 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:52:37 dist-test-slave-c51v krb5kdc[26462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697157, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:52:37.230264 26835 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:52:37.231808 26835 webserver.cc:460] Webserver started at http://127.15.196.195:41947/ using document root <none> and password file <none>
I20260929 15:52:37.232527 26835 fs_manager.cc:359] Metadata directory not provided
I20260929 15:52:37.232635 26835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:52:37.232969 26835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:52:37.236006 26835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "7b123b194a3a416b83e1e1854029cee8"
format_stamp: "Formatted at 2026-09-29 15:52:37 on dist-test-slave-c51v"
server_key: "22db4fd863d6af5c63075cfba3b96f46"
server_key_iv: "0654bbbdb8bc28cc661df886c0b2b094"
server_key_version: "encryptionkey@0"
I20260929 15:52:37.236883 26835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "7b123b194a3a416b83e1e1854029cee8"
format_stamp: "Formatted at 2026-09-29 15:52:37 on dist-test-slave-c51v"
server_key: "22db4fd863d6af5c63075cfba3b96f46"
server_key_iv: "0654bbbdb8bc28cc661df886c0b2b094"
server_key_version: "encryptionkey@0"
I20260929 15:52:37.242039 26835 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 15:52:37.245723 26850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:37.247164 26835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 15:52:37.247372 26835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "7b123b194a3a416b83e1e1854029cee8"
format_stamp: "Formatted at 2026-09-29 15:52:37 on dist-test-slave-c51v"
server_key: "22db4fd863d6af5c63075cfba3b96f46"
server_key_iv: "0654bbbdb8bc28cc661df886c0b2b094"
server_key_version: "encryptionkey@0"
I20260929 15:52:37.247548 26835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:52:37.262730 26835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:52:37.267232 26835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:52:37.267580 26835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:52:37.268570 26835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:52:37.269992 26835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:52:37.270066 26835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:37.270244 26835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:52:37.270298 26835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:52:37.304201 26835 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:39537
I20260929 15:52:37.304296 26962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:39537 every 8 connection(s)
I20260929 15:52:37.305454 26835 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:52:37.313865 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 26835
I20260929 15:52:37.314075 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:52:37.314479 16147 external_mini_cluster.cc:1428] Setting key 08f165f249fc8576492d76d18993456c
Sep 29 15:52:37 dist-test-slave-c51v krb5kdc[26462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697157, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:52:37.324734 26698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:37.308020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:55099 (local address 127.15.196.254:35139)
0929 15:52:37.308232 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:37.308238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:37.309363 (+ 1125us) server_negotiation.cc:407] Connection header received
0929 15:52:37.311089 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:37.311094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:37.311168 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:37.311257 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:37.312736 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.313512 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.315157 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.315774 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.315912 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:37.319200 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:37.319207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:37.319211 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:37.319252 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:37.321711 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:37.322396 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:37.322402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:37.322405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:37.322536 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:37.322958 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:37.322964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:37.322967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:37.323243 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:37.323438 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:37.324191 (+ 753us) server_negotiation.cc:299] Negotiation successful
0929 15:52:37.324399 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 15:52:37.325623 26965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:37.308499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:35139 (local address 127.15.196.195:55099)
0929 15:52:37.309208 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:37.309247 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:37.310829 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:37.311513 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:37.311521 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:37.312036 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:52:37.312582 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:37.312598 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.313650 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.313653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:37.315018 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:37.315029 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.315995 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.316003 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:37.316255 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:37.316912 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:52:37.316933 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:52:37.318993 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:52:37.321854 (+ 2861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:37.321863 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:37.321879 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:37.322274 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:37.322651 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:52:37.322656 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:52:37.322659 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:52:37.322846 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:52:37.323398 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:52:37.323406 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:52:37.323833 (+ 427us) client_negotiation.cc:769] Sending connection context
0929 15:52:37.324095 (+ 262us) client_negotiation.cc:240] Negotiation successful
0929 15:52:37.324589 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":266,"threads_started":1}
I20260929 15:52:37.326987 26963 heartbeater.cc:347] Connected to a master server at 127.15.196.254:35139
I20260929 15:52:37.327342 26963 heartbeater.cc:464] Registering TS with master...
I20260929 15:52:37.328253 26963 heartbeater.cc:511] Master 127.15.196.254:35139 requested a full tablet report, sending...
I20260929 15:52:37.329699 26507 ts_manager.cc:194] Registered new tserver with Master: 7b123b194a3a416b83e1e1854029cee8 (127.15.196.195:39537)
I20260929 15:52:37.330446 26507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:55099
I20260929 15:52:37.332248 26696 heartbeater.cc:503] Master 127.15.196.254:35139 was elected leader, sending a full tablet report...
I20260929 15:52:37.341390 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:52:37.358549 26698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:37.346203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41248 (local address 127.15.196.254:35139)
0929 15:52:37.346359 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:37.346364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:37.346916 (+ 552us) server_negotiation.cc:407] Connection header received
0929 15:52:37.347111 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:37.347136 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:37.347208 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:37.347401 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:37.348889 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.349722 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.350954 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.351449 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.351538 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:37.353855 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:37.353860 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:37.353863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:37.353892 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:37.356255 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:37.356876 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:37.356880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:37.356882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:37.357006 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:37.357313 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:37.357316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:37.357318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:37.357613 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:37.357804 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:37.358168 (+ 364us) server_negotiation.cc:299] Negotiation successful
0929 15:52:37.358318 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 15:52:37.363550 26507 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:52:37.370501 26507 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:52:37.405346 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:37.382998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33922 (local address 127.15.196.195:39537)
0929 15:52:37.396698 (+ 13700us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:37.396706 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:37.396740 (+ 34us) server_negotiation.cc:407] Connection header received
0929 15:52:37.396825 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:37.396831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:37.397034 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:37.397231 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:52:37.398660 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.400054 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.402312 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.403605 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.403807 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:37.404000 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:37.404895 (+ 895us) server_negotiation.cc:299] Negotiation successful
0929 15:52:37.405051 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13517,"thread_start_us":14005,"threads_started":1}
I20260929 15:52:37.405390 26976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:37.382251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:45103 (local address 127.0.0.1:44484)
0929 15:52:37.394847 (+ 12596us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:37.394890 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:37.396003 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:37.397419 (+ 1416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:37.397426 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:37.397461 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:52:37.398077 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:37.398091 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.400122 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.400130 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:37.402248 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:37.402266 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.404172 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.404177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:37.404478 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:37.404605 (+ 127us) client_negotiation.cc:769] Sending connection context
0929 15:52:37.404740 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 15:52:37.404953 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12290,"mutex_wait_us":260,"thread_start_us":15685,"threads_started":1}
I20260929 15:52:37.405359 26977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:37.382572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44484 (local address 127.15.196.193:45103)
0929 15:52:37.383223 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:37.383234 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:37.395039 (+ 11805us) server_negotiation.cc:407] Connection header received
0929 15:52:37.396559 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:37.396569 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:37.396873 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:37.397053 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:52:37.398283 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.399959 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.402399 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.403893 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.404087 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:37.404267 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:37.404821 (+ 554us) server_negotiation.cc:299] Negotiation successful
0929 15:52:37.404999 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":996,"threads_started":1}
I20260929 15:52:37.406194 26979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:37.382442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:39537 (local address 127.0.0.1:33922)
0929 15:52:37.395855 (+ 13413us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:37.395874 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:37.396002 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:37.397486 (+ 1484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:37.397492 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:37.397518 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:52:37.398463 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:37.398475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.400270 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.400275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:37.402143 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:37.402156 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.404147 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.404158 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:37.404436 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:37.404605 (+ 169us) client_negotiation.cc:769] Sending connection context
0929 15:52:37.404740 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 15:52:37.405894 (+ 1154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13242,"mutex_wait_us":1514,"thread_start_us":11971,"threads_started":1}
I20260929 15:52:37.406875 26975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:37.383450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:36491 (local address 127.0.0.1:34414)
0929 15:52:37.398296 (+ 14846us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:37.398316 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:37.398463 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:37.399314 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:37.399319 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:37.399343 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:52:37.399771 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:37.399781 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.401656 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.401662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:37.403884 (+ 2222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:37.403898 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.406170 (+ 2272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.406176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:37.406335 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:37.406366 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 15:52:37.406438 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 15:52:37.406554 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14702,"thread_start_us":12083,"threads_started":1}
I20260929 15:52:37.407698 26980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:37.383665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34414 (local address 127.15.196.194:36491)
0929 15:52:37.397706 (+ 14041us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:37.397758 (+ 52us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:37.398443 (+ 685us) server_negotiation.cc:407] Connection header received
0929 15:52:37.398610 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:37.398617 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:37.398878 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:37.399068 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:52:37.399952 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.401496 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.404068 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.405380 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.405543 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:37.405725 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:37.407153 (+ 1428us) server_negotiation.cc:299] Negotiation successful
0929 15:52:37.407347 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13897,"thread_start_us":13788,"threads_started":1}
I20260929 15:52:37.410044 26898 tablet_service.cc:1467] Processing CreateTablet for tablet 28cab0a9b31c4416a07fca724f941aa6 (DEFAULT_TABLE table=test-table [id=a00ccccc09a24a8b9acee983e4ed409d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:52:37.410043 26631 tablet_service.cc:1467] Processing CreateTablet for tablet 28cab0a9b31c4416a07fca724f941aa6 (DEFAULT_TABLE table=test-table [id=a00ccccc09a24a8b9acee983e4ed409d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:52:37.410725 26765 tablet_service.cc:1467] Processing CreateTablet for tablet 28cab0a9b31c4416a07fca724f941aa6 (DEFAULT_TABLE table=test-table [id=a00ccccc09a24a8b9acee983e4ed409d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:52:37.411604 26898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28cab0a9b31c4416a07fca724f941aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:37.411604 26631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28cab0a9b31c4416a07fca724f941aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:37.411923 26765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28cab0a9b31c4416a07fca724f941aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:52:37.424854 26981 tablet_bootstrap.cc:492] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058: Bootstrap starting.
I20260929 15:52:37.427966 26982 tablet_bootstrap.cc:492] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a: Bootstrap starting.
I20260929 15:52:37.429157 26981 tablet_bootstrap.cc:654] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:37.429618 26983 tablet_bootstrap.cc:492] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8: Bootstrap starting.
I20260929 15:52:37.430552 26981 log.cc:824] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058: Log is configured to *not* fsync() on all Append() calls
I20260929 15:52:37.431529 26982 tablet_bootstrap.cc:654] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:37.432756 26983 tablet_bootstrap.cc:654] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8: Neither blocks nor log segments found. Creating new log.
I20260929 15:52:37.432740 26982 log.cc:824] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a: Log is configured to *not* fsync() on all Append() calls
I20260929 15:52:37.433804 26981 tablet_bootstrap.cc:492] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058: No bootstrap required, opened a new log
I20260929 15:52:37.433988 26983 log.cc:824] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8: Log is configured to *not* fsync() on all Append() calls
I20260929 15:52:37.434108 26981 ts_tablet_manager.cc:1397] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260929 15:52:37.436081 26982 tablet_bootstrap.cc:492] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a: No bootstrap required, opened a new log
I20260929 15:52:37.436347 26982 ts_tablet_manager.cc:1397] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 15:52:37.436885 26983 tablet_bootstrap.cc:492] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8: No bootstrap required, opened a new log
I20260929 15:52:37.437134 26983 ts_tablet_manager.cc:1397] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260929 15:52:37.444276 26983 raft_consensus.cc:348] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b65b0f8fe7e4a7292ed597d67edce6a" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45103 } } peers { permanent_uuid: "7b123b194a3a416b83e1e1854029cee8" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39537 } } peers { permanent_uuid: "61daba3d1f4047a0b6fe96d7827df058" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 36491 } }
I20260929 15:52:37.444839 26983 raft_consensus.cc:374] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:37.444952 26983 raft_consensus.cc:729] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b123b194a3a416b83e1e1854029cee8, State: Initialized, Role: FOLLOWER
I20260929 15:52:37.444844 26981 raft_consensus.cc:348] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b65b0f8fe7e4a7292ed597d67edce6a" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45103 } } peers { permanent_uuid: "7b123b194a3a416b83e1e1854029cee8" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39537 } } peers { permanent_uuid: "61daba3d1f4047a0b6fe96d7827df058" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 36491 } }
I20260929 15:52:37.445266 26981 raft_consensus.cc:374] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:37.445360 26981 raft_consensus.cc:729] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61daba3d1f4047a0b6fe96d7827df058, State: Initialized, Role: FOLLOWER
I20260929 15:52:37.445713 26983 consensus_queue.cc:260] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b65b0f8fe7e4a7292ed597d67edce6a" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45103 } } peers { permanent_uuid: "7b123b194a3a416b83e1e1854029cee8" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39537 } } peers { permanent_uuid: "61daba3d1f4047a0b6fe96d7827df058" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 36491 } }
I20260929 15:52:37.446158 26981 consensus_queue.cc:260] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b65b0f8fe7e4a7292ed597d67edce6a" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45103 } } peers { permanent_uuid: "7b123b194a3a416b83e1e1854029cee8" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39537 } } peers { permanent_uuid: "61daba3d1f4047a0b6fe96d7827df058" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 36491 } }
I20260929 15:52:37.446875 26982 raft_consensus.cc:348] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b65b0f8fe7e4a7292ed597d67edce6a" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45103 } } peers { permanent_uuid: "7b123b194a3a416b83e1e1854029cee8" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39537 } } peers { permanent_uuid: "61daba3d1f4047a0b6fe96d7827df058" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 36491 } }
I20260929 15:52:37.447448 26982 raft_consensus.cc:374] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:52:37.447571 26982 raft_consensus.cc:729] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b65b0f8fe7e4a7292ed597d67edce6a, State: Initialized, Role: FOLLOWER
I20260929 15:52:37.448351 26982 consensus_queue.cc:260] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b65b0f8fe7e4a7292ed597d67edce6a" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45103 } } peers { permanent_uuid: "7b123b194a3a416b83e1e1854029cee8" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39537 } } peers { permanent_uuid: "61daba3d1f4047a0b6fe96d7827df058" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 36491 } }
I20260929 15:52:37.450560 26982 ts_tablet_manager.cc:1428] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260929 15:52:37.452171 26963 heartbeater.cc:503] Master 127.15.196.254:35139 was elected leader, sending a full tablet report...
I20260929 15:52:37.452756 26983 ts_tablet_manager.cc:1428] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8: Time spent starting tablet: real 0.015s user 0.010s sys 0.006s
I20260929 15:52:37.453529 26830 heartbeater.cc:503] Master 127.15.196.254:35139 was elected leader, sending a full tablet report...
I20260929 15:52:37.454430 26981 ts_tablet_manager.cc:1428] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058: Time spent starting tablet: real 0.020s user 0.014s sys 0.004s
W20260929 15:52:37.551796 26697 tablet.cc:2369] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 15:52:37.553140 26831 tablet.cc:2369] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 15:52:37.556974 26964 tablet.cc:2369] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:52:37.569617 26988 raft_consensus.cc:482] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:52:37.569969 26988 raft_consensus.cc:504] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b65b0f8fe7e4a7292ed597d67edce6a" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45103 } } peers { permanent_uuid: "7b123b194a3a416b83e1e1854029cee8" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39537 } } peers { permanent_uuid: "61daba3d1f4047a0b6fe96d7827df058" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 36491 } }
I20260929 15:52:37.571862 26988 leader_election.cc:288] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b65b0f8fe7e4a7292ed597d67edce6a (127.15.196.193:45103), 7b123b194a3a416b83e1e1854029cee8 (127.15.196.195:39537)
I20260929 15:52:37.578692 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:37.572445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:50913 (local address 127.15.196.195:39537)
0929 15:52:37.572665 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:37.572672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:37.573032 (+ 360us) server_negotiation.cc:407] Connection header received
0929 15:52:37.573284 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:37.573290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:37.573372 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:37.573477 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:52:37.574078 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.575002 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.576881 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.577705 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.577860 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:37.577949 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:37.578297 (+ 348us) server_negotiation.cc:299] Negotiation successful
0929 15:52:37.578432 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 15:52:37.578727 26977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:37.572287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:39557 (local address 127.15.196.193:45103)
0929 15:52:37.572538 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:37.572546 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:37.573575 (+ 1029us) server_negotiation.cc:407] Connection header received
0929 15:52:37.573770 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:37.573775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:37.573851 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:37.573950 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:52:37.574547 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.575347 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.577030 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.577762 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.577940 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:37.578011 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:37.578364 (+ 353us) server_negotiation.cc:299] Negotiation successful
0929 15:52:37.578487 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 15:52:37.578819 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:37.572181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:39537 (local address 127.15.196.194:50913)
0929 15:52:37.572884 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:37.572905 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:37.573092 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:37.573500 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:37.573503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:37.573546 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:52:37.573935 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:37.573947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.575124 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.575130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:37.576733 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:37.576745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.577928 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.577934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:37.578126 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:37.578170 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 15:52:37.578240 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 15:52:37.578369 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":4,"thread_start_us":322,"threads_started":1}
I20260929 15:52:37.578876 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:37.572184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:45103 (local address 127.15.196.194:39557)
0929 15:52:37.573467 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0929 15:52:37.573483 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 15:52:37.573607 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:52:37.573968 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:52:37.573971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:52:37.573990 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:52:37.574400 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:37.574413 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.575472 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.575476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:37.576881 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:52:37.576895 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.578007 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.578012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:52:37.578188 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:37.578210 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 15:52:37.578318 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 15:52:37.578433 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1156,"mutex_wait_us":612,"thread_start_us":369,"threads_started":1}
I20260929 15:52:37.579998 26651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28cab0a9b31c4416a07fca724f941aa6" candidate_uuid: "61daba3d1f4047a0b6fe96d7827df058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b65b0f8fe7e4a7292ed597d67edce6a" is_pre_election: true
I20260929 15:52:37.580219 26918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28cab0a9b31c4416a07fca724f941aa6" candidate_uuid: "61daba3d1f4047a0b6fe96d7827df058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b123b194a3a416b83e1e1854029cee8" is_pre_election: true
I20260929 15:52:37.580606 26651 raft_consensus.cc:2454] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61daba3d1f4047a0b6fe96d7827df058 in term 0.
I20260929 15:52:37.580701 26918 raft_consensus.cc:2454] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61daba3d1f4047a0b6fe96d7827df058 in term 0.
I20260929 15:52:37.581336 26718 leader_election.cc:302] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61daba3d1f4047a0b6fe96d7827df058, 7b65b0f8fe7e4a7292ed597d67edce6a; no voters:
I20260929 15:52:37.581732 26988 raft_consensus.cc:2781] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:52:37.581878 26988 raft_consensus.cc:482] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:52:37.581981 26988 raft_consensus.cc:3037] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:37.584161 26988 raft_consensus.cc:504] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b65b0f8fe7e4a7292ed597d67edce6a" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45103 } } peers { permanent_uuid: "7b123b194a3a416b83e1e1854029cee8" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39537 } } peers { permanent_uuid: "61daba3d1f4047a0b6fe96d7827df058" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 36491 } }
I20260929 15:52:37.584781 26988 leader_election.cc:288] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [CANDIDATE]: Term 1 election: Requested vote from peers 7b65b0f8fe7e4a7292ed597d67edce6a (127.15.196.193:45103), 7b123b194a3a416b83e1e1854029cee8 (127.15.196.195:39537)
I20260929 15:52:37.585187 26651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28cab0a9b31c4416a07fca724f941aa6" candidate_uuid: "61daba3d1f4047a0b6fe96d7827df058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b65b0f8fe7e4a7292ed597d67edce6a"
I20260929 15:52:37.585325 26918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28cab0a9b31c4416a07fca724f941aa6" candidate_uuid: "61daba3d1f4047a0b6fe96d7827df058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b123b194a3a416b83e1e1854029cee8"
I20260929 15:52:37.585426 26651 raft_consensus.cc:3037] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:37.585520 26918 raft_consensus.cc:3037] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:52:37.588753 26651 raft_consensus.cc:2454] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61daba3d1f4047a0b6fe96d7827df058 in term 1.
I20260929 15:52:37.588740 26918 raft_consensus.cc:2454] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61daba3d1f4047a0b6fe96d7827df058 in term 1.
I20260929 15:52:37.589205 26718 leader_election.cc:302] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61daba3d1f4047a0b6fe96d7827df058, 7b65b0f8fe7e4a7292ed597d67edce6a; no voters:
I20260929 15:52:37.589530 26988 raft_consensus.cc:2781] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:52:37.590019 26988 raft_consensus.cc:686] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [term 1 LEADER]: Becoming Leader. State: Replica: 61daba3d1f4047a0b6fe96d7827df058, State: Running, Role: LEADER
I20260929 15:52:37.590621 26988 consensus_queue.cc:237] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b65b0f8fe7e4a7292ed597d67edce6a" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45103 } } peers { permanent_uuid: "7b123b194a3a416b83e1e1854029cee8" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39537 } } peers { permanent_uuid: "61daba3d1f4047a0b6fe96d7827df058" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 36491 } }
I20260929 15:52:37.595660 26506 catalog_manager.cc:5495] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 reported cstate change: term changed from 0 to 1, leader changed from <none> to 61daba3d1f4047a0b6fe96d7827df058 (127.15.196.194). New cstate: current_term: 1 leader_uuid: "61daba3d1f4047a0b6fe96d7827df058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b65b0f8fe7e4a7292ed597d67edce6a" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b123b194a3a416b83e1e1854029cee8" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61daba3d1f4047a0b6fe96d7827df058" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 36491 } health_report { overall_health: HEALTHY } } }
I20260929 15:52:37.645506 26980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:37.639907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34430 (local address 127.15.196.194:36491)
0929 15:52:37.640122 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:37.640129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:37.640226 (+ 97us) server_negotiation.cc:407] Connection header received
0929 15:52:37.640475 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:37.640479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:37.640561 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:37.640713 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:52:37.641080 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.641823 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.643197 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.643732 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.643883 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:37.644417 (+ 534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:52:37.645038 (+ 621us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:52:37.645183 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:37.645236 (+ 53us) server_negotiation.cc:299] Negotiation successful
0929 15:52:37.645300 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 15:52:37.653838 26918 raft_consensus.cc:1261] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8 [term 1 FOLLOWER]: Refusing update from remote peer 61daba3d1f4047a0b6fe96d7827df058: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:52:37.653838 26651 raft_consensus.cc:1261] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a [term 1 FOLLOWER]: Refusing update from remote peer 61daba3d1f4047a0b6fe96d7827df058: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:52:37.654909 26992 consensus_queue.cc:1035] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b65b0f8fe7e4a7292ed597d67edce6a" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:52:37.655258 26988 consensus_queue.cc:1035] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b123b194a3a416b83e1e1854029cee8" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:52:37.664260 26995 mvcc.cc:205] Tried to move back new op lower bound from 7334695557739700224 to 7334695557495238656. Current Snapshot: MvccSnapshot[applied={T|T < 7334695557739700224}]
I20260929 15:52:37.679701 26507 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:52:37.679991 26507 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:52:37.683480 26507 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2e749b75b07403fb5daadb937ea2d63: Sending DeleteTablet for 3 replicas of tablet 28cab0a9b31c4416a07fca724f941aa6
I20260929 15:52:37.684719 26898 tablet_service.cc:1514] Processing DeleteTablet for tablet 28cab0a9b31c4416a07fca724f941aa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:52:37 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:33922
I20260929 15:52:37.684788 26765 tablet_service.cc:1514] Processing DeleteTablet for tablet 28cab0a9b31c4416a07fca724f941aa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:52:37 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:34414
I20260929 15:52:37.684835 26631 tablet_service.cc:1514] Processing DeleteTablet for tablet 28cab0a9b31c4416a07fca724f941aa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:52:37 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:44484
I20260929 15:52:37.689579 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 26568
I20260929 15:52:37.697939 27004 tablet_replica.cc:330] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8: stopping tablet replica
I20260929 15:52:37.698186 27005 tablet_replica.cc:330] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058: stopping tablet replica
I20260929 15:52:37.698647 27005 raft_consensus.cc:2229] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:52:37.699352 27005 raft_consensus.cc:2258] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:52:37.699826 27004 raft_consensus.cc:2229] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:52:37.700453 27004 raft_consensus.cc:2258] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:52:37.702255 27006 tablet_replica.cc:330] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a: stopping tablet replica
I20260929 15:52:37.702736 27004 ts_tablet_manager.cc:1905] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:52:37.702700 27006 raft_consensus.cc:2229] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:52:37.703203 27006 raft_consensus.cc:2258] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:52:37.705163 27006 ts_tablet_manager.cc:1905] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:52:37.706769 27005 ts_tablet_manager.cc:1905] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 29 15:52:37 dist-test-slave-c51v krb5kdc[26462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:52:37.710294 27004 ts_tablet_manager.cc:1918] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:52:37.710538 27004 log.cc:1196] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/28cab0a9b31c4416a07fca724f941aa6
I20260929 15:52:37.711151 27004 ts_tablet_manager.cc:1939] T 28cab0a9b31c4416a07fca724f941aa6 P 7b123b194a3a416b83e1e1854029cee8: Deleting consensus metadata
I20260929 15:52:37.712410 27006 ts_tablet_manager.cc:1918] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:52:37.712654 27006 log.cc:1196] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/28cab0a9b31c4416a07fca724f941aa6
I20260929 15:52:37.713253 27006 ts_tablet_manager.cc:1939] T 28cab0a9b31c4416a07fca724f941aa6 P 7b65b0f8fe7e4a7292ed597d67edce6a: Deleting consensus metadata
I20260929 15:52:37.713256 26493 catalog_manager.cc:4841] TS 7b123b194a3a416b83e1e1854029cee8 (127.15.196.195:39537): tablet 28cab0a9b31c4416a07fca724f941aa6 (table test-table [id=a00ccccc09a24a8b9acee983e4ed409d]) successfully deleted
I20260929 15:52:37.715140 26491 catalog_manager.cc:4841] TS 7b65b0f8fe7e4a7292ed597d67edce6a (127.15.196.193:45103): tablet 28cab0a9b31c4416a07fca724f941aa6 (table test-table [id=a00ccccc09a24a8b9acee983e4ed409d]) successfully deleted
I20260929 15:52:37.719327 27005 ts_tablet_manager.cc:1918] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:52:37.719583 27005 log.cc:1196] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/28cab0a9b31c4416a07fca724f941aa6
I20260929 15:52:37.720141 26977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:37.690852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44496 (local address 127.15.196.193:45103)
0929 15:52:37.691046 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:37.691054 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:37.693722 (+ 2668us) server_negotiation.cc:407] Connection header received
0929 15:52:37.693963 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:37.693969 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:37.694052 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:37.694166 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:37.701730 (+ 7564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.703036 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.705005 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.705697 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.705802 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:37.713971 (+ 8169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:37.713984 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:37.714001 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:37.714050 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:37.717270 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:37.718042 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:37.718049 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:37.718051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:37.718189 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:37.718581 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:37.718586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:37.718588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:37.718927 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:37.719248 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:37.719554 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 15:52:37.719777 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 15:52:37.720373 27005 ts_tablet_manager.cc:1939] T 28cab0a9b31c4416a07fca724f941aa6 P 61daba3d1f4047a0b6fe96d7827df058: Deleting consensus metadata
I20260929 15:52:37.720834 26691 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:37.722393 26492 catalog_manager.cc:4841] TS 61daba3d1f4047a0b6fe96d7827df058 (127.15.196.194:36491): tablet 28cab0a9b31c4416a07fca724f941aa6 (table test-table [id=a00ccccc09a24a8b9acee983e4ed409d]) successfully deleted
I20260929 15:52:37.835212 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 26568
I20260929 15:52:37.853309 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 26702
Sep 29 15:52:37 dist-test-slave-c51v krb5kdc[26462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:52:37.870967 26980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:37.854144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34446 (local address 127.15.196.194:36491)
0929 15:52:37.854297 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:37.854301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:37.854363 (+ 62us) server_negotiation.cc:407] Connection header received
0929 15:52:37.854572 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:37.854576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:37.854668 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:37.854747 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:37.856409 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.857196 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.859108 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:37.859575 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:37.859666 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:37.864581 (+ 4915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:37.864596 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:37.864613 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:37.864651 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:37.867824 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:37.868531 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:37.868539 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:37.868541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:37.868660 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:37.869053 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:37.869059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:37.869062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:37.869576 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:37.869931 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:37.870258 (+ 327us) server_negotiation.cc:299] Negotiation successful
0929 15:52:37.870477 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 15:52:37.871743 26825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:37.985342 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 26702
I20260929 15:52:38.003535 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 26835
Sep 29 15:52:38 dist-test-slave-c51v krb5kdc[26462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:52:38.018085 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:52:38.004352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33928 (local address 127.15.196.195:39537)
0929 15:52:38.004530 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 15:52:38.004535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:52:38.004553 (+ 18us) server_negotiation.cc:407] Connection header received
0929 15:52:38.004693 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:52:38.004697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:52:38.004758 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:52:38.004839 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:52:38.006133 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:38.006838 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:38.008196 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:52:38.008560 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:52:38.008650 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:52:38.012882 (+ 4232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:52:38.012895 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:52:38.012911 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:52:38.012946 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:52:38.015648 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:38.016246 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:38.016253 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:38.016255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:38.016391 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:52:38.016700 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:52:38.016703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:52:38.016705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:52:38.017022 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:52:38.017250 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 15:52:38.017500 (+ 250us) server_negotiation.cc:299] Negotiation successful
0929 15:52:38.017663 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 15:52:38.018745 26958 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:38.109200 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 26835
I20260929 15:52:38.125933 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 26475
I20260929 15:52:38.127111 26537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:52:38.237279 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 26475
2026-09-29T15:52:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4474 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:52:38 dist-test-slave-c51v krb5kdc[27014](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:52:38 dist-test-slave-c51v krb5kdc[27014](info): set up 2 sockets
Sep 29 15:52:38 dist-test-slave-c51v krb5kdc[27014](info): commencing operation
krb5kdc: starting...
W20260929 15:52:39.338330 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:52:39 dist-test-slave-c51v krb5kdc[27014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:52:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:52:39Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/rangeradmin_127.15.196.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/rangeradmin_127.15.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/rangerlookup_127.15.196.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/rangerlookup_127.15.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/HTTP_127.15.196.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/HTTP_127.15.196.212@KRBTEST.COM.keytab.
I20260929 15:52:39.664763 16147 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task6fGZba/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/postgres -l logfile start
2026-09-29 15:52:42.214 UTC [27044] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 15:52:42.214 UTC [27044] LOG: listening on IPv4 address "127.15.196.212", port 57095
2026-09-29 15:52:42.218 UTC [27044] LOG: listening on Unix socket "/tmp/.s.PGSQL.57095"
2026-09-29 15:52:42.242 UTC [27047] LOG: database system was shut down at 2026-09-29 15:52:40 UTC
2026-09-29 15:52:42.246 UTC [27044] LOG: database system is ready to accept connections
I20260929 15:52:43.221942 16147 mini_postgres.cc:96] Postgres bound to 57095
2026-09-29 15:52:43.240 UTC [27055] FATAL: database "slave" does not exist
127.15.196.212:57095 - accepting connections
I20260929 15:52:43.240998 16147 mini_ranger.cc:161] Starting Ranger...
I20260929 15:52:43.267378 16147 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 15:52:43.747761 16147 mini_ranger.cc:87] Created ranger Postgres database
I20260929 15:52:43.747897 16147 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-admin
2026-09-29 15:52:44,035 [I] DB FLAVOR :POSTGRES
2026-09-29 15:52:44,036 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 15:52:44,036 [I] Checking connection..
2026-09-29 15:52:44,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 15:52:44,301 [I] Checking connection passed.
2026-09-29 15:52:44,301 [I] --------- Verifying version history table ---------
2026-09-29 15:52:44,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 15:52:44,547 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 15:52:44,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 15:52:44,808 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 15:52:44,808 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 15:52:44,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 15:52:45.058 UTC [27119] WARNING: there is no transaction in progress
2026-09-29 15:52:45,065 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 15:52:45,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 15:52:45,334 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 15:52:45,335 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 15:52:45,335 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 15:52:45,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 15:52:45,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-c51v.c.gcp-upstream.internal','N') ;"
2026-09-29 15:52:46,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 15:52:46,345 [I] Table x_portal_user does not exist in database ranger
2026-09-29 15:52:46,345 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 15:52:46,584 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 15:52:46,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 15:52:46,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 15:52:47,067 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 15:52:47,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 15:52:48.062 UTC [27227] WARNING: there is no transaction in progress
2026-09-29 15:52:48.078 UTC [27227] WARNING: there is no transaction in progress
2026-09-29 15:52:48.094 UTC [27227] WARNING: there is no transaction in progress
2026-09-29 15:52:48.111 UTC [27227] WARNING: there is no transaction in progress
2026-09-29 15:52:48.127 UTC [27227] WARNING: there is no transaction in progress
2026-09-29 15:52:48.149 UTC [27227] WARNING: there is no transaction in progress
2026-09-29 15:52:48.204 UTC [27227] WARNING: there is no transaction in progress
2026-09-29 15:52:48.211 UTC [27227] WARNING: there is no transaction in progress
2026-09-29 15:52:48.226 UTC [27227] WARNING: there is no transaction in progress
2026-09-29 15:52:48.237 UTC [27227] WARNING: there is no transaction in progress
2026-09-29 15:52:48.248 UTC [27227] WARNING: there is no transaction in progress
2026-09-29 15:52:48.262 UTC [27227] WARNING: there is no transaction in progress
2026-09-29 15:52:48.273 UTC [27227] WARNING: there is no transaction in progress
2026-09-29 15:52:48.280 UTC [27227] WARNING: there is no transaction in progress
2026-09-29 15:52:48.286 UTC [27227] WARNING: there is no transaction in progress
2026-09-29 15:52:48.293 UTC [27227] WARNING: there is no transaction in progress
2026-09-29 15:52:48,937 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 15:52:48,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 15:52:49,200 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 15:52:49,201 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 15:52:49,456 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 15:52:49,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 15:52:49,697 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 15:52:49,697 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 15:52:49,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 15:52:50,185 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-c51v.c.gcp-upstream.internal';"
2026-09-29 15:52:50,425 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 15:52:50,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 15:52:50,678 [I] DB_PATCHES have already been applied
I20260929 15:52:50.684706 16147 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-admin:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task6fGZba/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 15:52:50.684859 16147 mini_ranger.cc:193] Using host: 127.15.196.212
I20260929 15:52:50.700426 16147 mini_ranger.cc:227] Ranger admin URL: http://127.15.196.212:47437
Sep 29, 2026 3:52:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 3:52:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 3:52:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 3:52:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 3:52:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 3:52:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 3:52:51 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 15:52:51.835577 16147 mini_ranger.cc:160] Time spent starting Ranger: real 8.595s user 0.011s sys 0.032s
Sep 29, 2026 3:53:12 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 3:53:12 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 29, 2026 3:53:12 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 3:53:12 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 3:53:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 3:53:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 3:53:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 3:53:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 3:53:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 3:53:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 3:53:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 3:53:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 3:53:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 3:53:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 3:53:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 3:53:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 3:53:12 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 3:53:13 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 15:53:13.823836 16147 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/rangerkms_127.15.196.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/rangerkms_127.15.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.196.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.196.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.196.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/HTTP_127.15.196.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.196.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/HTTP_127.15.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 15:53:14 dist-test-slave-c51v krb5kdc[27014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697194, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 15:53:14.020368 16147 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 15:53:14.045056 16147 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 15:53:14.959424 16147 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 15:53:14.959583 16147 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-kms
I20260929 15:53:14.959697 16147 mini_ranger_kms.cc:227] Using postgres at 127.15.196.212:57095
2026-09-29 15:53:15,398 [I] DB FLAVOR :POSTGRES
2026-09-29 15:53:15,399 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 15:53:15,399 [I] Checking connection
2026-09-29 15:53:15,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 15:53:15,632 [I] connection success
2026-09-29 15:53:15,632 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 15:53:15,632 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 15:53:15,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 15:53:15,878 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 15:53:15,878 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 15:53:15,878 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 15:53:15,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:57095/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 15:53:16,145 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 15:53:16.635196 16147 mini_ranger_kms.cc:310] Created kms service
I20260929 15:53:16.846648 16147 mini_ranger_kms.cc:326] Created kudu user
I20260929 15:53:16.892439 16147 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 15:53:16 dist-test-slave-c51v krb5kdc[27014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697194, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
I20260929 15:53:17.325714 16147 mini_ranger_kms.cc:384] Added ranger policy
I20260929 15:53:17.326071 16147 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-kms:/tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 15:53:17.326138 16147 mini_ranger_kms.cc:240] Using host: 127.15.196.212
I20260929 15:53:17.343631 16147 mini_ranger_kms.cc:276] Ranger KMS PID: 27407
I20260929 15:53:17.343845 16147 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.196.212:38791
Sep 29, 2026 3:53:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 3:53:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 3:53:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 3:53:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 3:53:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 3:53:19 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 15:53:19.721971 16147 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.702s user 0.129s sys 0.330s
I20260929 15:53:19.722240 16147 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 15:53:19.722469 16147 mini_ranger_kms.cc:401] 127.15.196.212:38791/kms/v1/keys
Sep 29 15:53:19 dist-test-slave-c51v krb5kdc[27014](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790697199, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.196.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 15:53:20 dist-test-slave-c51v krb5kdc[27014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697199, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.196.212@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
Sep 29 15:53:20 dist-test-slave-c51v krb5kdc[27014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697199, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.196.212@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
Sep 29 15:53:23 dist-test-slave-c51v krb5kdc[27014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:53:23.211851 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:46497
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:46497
--ranger_config_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.196.212:38791/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:53:23.557384 27458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:53:23.557685 27458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:53:23.557857 27458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:53:23.565500 27458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:53:23.565616 27458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:53:23.565718 27458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:53:23.565819 27458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:53:23.565900 27458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:53:23.576285 27458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.196.212:38791/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:46497
--ranger_config_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:46497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:53:23.577625 27458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:53:23.579105 27458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:53:23.590982 27466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:53:23.591130 27463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:53:23.591144 27464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:53:23.591739 27458 server_base.cc:1034] running on GCE node
I20260929 15:53:23.592494 27458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:53:23.593876 27458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:53:23.595079 27458 hybrid_clock.cc:648] HybridClock initialized: now 1790697203595046 us; error 29 us; skew 500 ppm
I20260929 15:53:23.595482 27458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:53:23 dist-test-slave-c51v krb5kdc[27014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697203, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:53:23.598520 27458 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:53:23.600135 27458 webserver.cc:460] Webserver started at http://127.15.196.254:33451/ using document root <none> and password file <none>
I20260929 15:53:23.600792 27458 fs_manager.cc:359] Metadata directory not provided
I20260929 15:53:23.600936 27458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:53:23.601234 27458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 15:53:23 dist-test-slave-c51v krb5kdc[27014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697203, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
I20260929 15:53:23.660482 27458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "7d9f361c1e2b466bada636bed10c9f03"
format_stamp: "Formatted at 2026-09-29 15:53:23 on dist-test-slave-c51v"
server_key: "02ad4c430eed861b75485817215a0cec"
server_key_iv: "cb5965ec17e02ef1bf84ff215ffc4344"
server_key_version: "kuduclusterkey@0"
I20260929 15:53:23.661410 27458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "7d9f361c1e2b466bada636bed10c9f03"
format_stamp: "Formatted at 2026-09-29 15:53:23 on dist-test-slave-c51v"
server_key: "02ad4c430eed861b75485817215a0cec"
server_key_iv: "cb5965ec17e02ef1bf84ff215ffc4344"
server_key_version: "kuduclusterkey@0"
I20260929 15:53:23.667235 27458 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 15:53:23.687395 27473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:53:23.689054 27458 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260929 15:53:23.689244 27458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "7d9f361c1e2b466bada636bed10c9f03"
format_stamp: "Formatted at 2026-09-29 15:53:23 on dist-test-slave-c51v"
server_key: "02ad4c430eed861b75485817215a0cec"
server_key_iv: "cb5965ec17e02ef1bf84ff215ffc4344"
server_key_version: "kuduclusterkey@0"
I20260929 15:53:23.689499 27458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:53:23.710395 27458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:53:23.714653 27458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:53:23.714994 27458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:53:23.735170 27458 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:46497
I20260929 15:53:23.735190 27524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:46497 every 8 connection(s)
I20260929 15:53:23.736810 27458 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:53:23.740952 27525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:53:23.742226 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 27458
I20260929 15:53:23.742447 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:53:23.743263 16147 external_mini_cluster.cc:1428] Setting key 2887666924c7ac315f62723d0b7026c6
I20260929 15:53:23.753166 27525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03: Bootstrap starting.
I20260929 15:53:23.756366 27525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03: Neither blocks nor log segments found. Creating new log.
I20260929 15:53:23.757371 27525 log.cc:824] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03: Log is configured to *not* fsync() on all Append() calls
Sep 29 15:53:23 dist-test-slave-c51v krb5kdc[27014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:53:23.761121 27525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03: No bootstrap required, opened a new log
I20260929 15:53:23.768031 27528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:23.745529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45838 (local address 127.15.196.254:46497)
0929 15:53:23.746887 (+ 1358us) server_negotiation.cc:206] Beginning negotiation
0929 15:53:23.746910 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 15:53:23.747304 (+ 394us) server_negotiation.cc:407] Connection header received
0929 15:53:23.749129 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:53:23.749151 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:53:23.749550 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:53:23.749939 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:53:23.751323 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:23.752616 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:23.753905 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:23.754566 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:23.754766 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:23.759154 (+ 4388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:53:23.759168 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:53:23.759188 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:53:23.759241 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:53:23.763241 (+ 4000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:53:23.763964 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:53:23.763976 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:53:23.763990 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:53:23.764204 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:53:23.764579 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:53:23.764586 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:53:23.764591 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:53:23.765364 (+ 773us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:53:23.765722 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0929 15:53:23.766056 (+ 334us) server_negotiation.cc:299] Negotiation successful
0929 15:53:23.766490 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1083,"thread_start_us":852,"threads_started":1}
I20260929 15:53:23.772225 27525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f361c1e2b466bada636bed10c9f03" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46497 } }
I20260929 15:53:23.772583 27525 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:53:23.772656 27525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d9f361c1e2b466bada636bed10c9f03, State: Initialized, Role: FOLLOWER
I20260929 15:53:23.773177 27525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f361c1e2b466bada636bed10c9f03" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46497 } }
I20260929 15:53:23.773411 27525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:53:23.773514 27525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:53:23.773648 27525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:53:23.776286 27525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f361c1e2b466bada636bed10c9f03" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46497 } }
I20260929 15:53:23.776753 27525 leader_election.cc:302] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d9f361c1e2b466bada636bed10c9f03; no voters:
I20260929 15:53:23.777331 27525 leader_election.cc:288] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:53:23.777477 27530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:53:23.778185 27530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [term 1 LEADER]: Becoming Leader. State: Replica: 7d9f361c1e2b466bada636bed10c9f03, State: Running, Role: LEADER
I20260929 15:53:23.778780 27530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f361c1e2b466bada636bed10c9f03" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46497 } }
I20260929 15:53:23.779659 27525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:53:23.782478 27531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d9f361c1e2b466bada636bed10c9f03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f361c1e2b466bada636bed10c9f03" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46497 } } }
I20260929 15:53:23.783025 27531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [sys.catalog]: This master's current role is: LEADER
I20260929 15:53:23.783897 27538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:53:23.784152 27532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d9f361c1e2b466bada636bed10c9f03. Latest consensus state: current_term: 1 leader_uuid: "7d9f361c1e2b466bada636bed10c9f03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f361c1e2b466bada636bed10c9f03" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 46497 } } }
I20260929 15:53:23.784574 27532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03 [sys.catalog]: This master's current role is: LEADER
I20260929 15:53:23.789366 27538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:53:23.791373 27525 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 15:53:23.809316 27538 catalog_manager.cc:1347] Generated new cluster ID: 989c599325be4fddad23c389e1ac74a1
I20260929 15:53:23.809473 27538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:53:23.842554 27538 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:53:23.844769 27538 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:53:23.865332 27538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03: Generated new TSK 0
I20260929 15:53:23.866590 27538 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:53:24.640175 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.196.212:38791/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46497
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 15:53:24 dist-test-slave-c51v krb5kdc[27014](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790697204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 15:53:25.060518 27570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:53:25.060878 27570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:53:25.061049 27570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:53:25.070019 27570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:53:25.070199 27570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:53:25.070389 27570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:53:25.081240 27570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.196.212:38791/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46497
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:53:25.082715 27570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:53:25.084116 27570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:53:25.103600 27580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:53:25.105158 27581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:53:25.113852 27570 server_base.cc:1034] running on GCE node
W20260929 15:53:25.114365 27583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:53:25.115254 27570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:53:25.116534 27570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:53:25.117861 27570 hybrid_clock.cc:648] HybridClock initialized: now 1790697205117766 us; error 59 us; skew 500 ppm
I20260929 15:53:25.118376 27570 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:53:25 dist-test-slave-c51v krb5kdc[27014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697205, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:53:25.122430 27570 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:53:25.124650 27570 webserver.cc:460] Webserver started at http://127.15.196.193:37185/ using document root <none> and password file <none>
I20260929 15:53:25.125411 27570 fs_manager.cc:359] Metadata directory not provided
I20260929 15:53:25.125550 27570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:53:25.125861 27570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 15:53:25 dist-test-slave-c51v krb5kdc[27014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697205, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
I20260929 15:53:25.152184 27570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "4c1783cedc324c49a6d1f599e145c483"
format_stamp: "Formatted at 2026-09-29 15:53:25 on dist-test-slave-c51v"
server_key: "649983b185bd38fa0163e83ec0e4cac1"
server_key_iv: "2778db0772ac1d88448941b42202b4fa"
server_key_version: "kuduclusterkey@0"
I20260929 15:53:25.155608 27570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "4c1783cedc324c49a6d1f599e145c483"
format_stamp: "Formatted at 2026-09-29 15:53:25 on dist-test-slave-c51v"
server_key: "649983b185bd38fa0163e83ec0e4cac1"
server_key_iv: "2778db0772ac1d88448941b42202b4fa"
server_key_version: "kuduclusterkey@0"
I20260929 15:53:25.162032 27570 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260929 15:53:25.191958 27590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:53:25.194484 27570 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.008s sys 0.008s
I20260929 15:53:25.194768 27570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "4c1783cedc324c49a6d1f599e145c483"
format_stamp: "Formatted at 2026-09-29 15:53:25 on dist-test-slave-c51v"
server_key: "649983b185bd38fa0163e83ec0e4cac1"
server_key_iv: "2778db0772ac1d88448941b42202b4fa"
server_key_version: "kuduclusterkey@0"
I20260929 15:53:25.195151 27570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:53:25.232298 27570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:53:25.237358 27570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:53:25.237727 27570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:53:25.240625 27570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:53:25.242409 27570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:53:25.242560 27570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:53:25.242758 27570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:53:25.243036 27570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:53:25.311549 27570 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:45335
I20260929 15:53:25.311604 27702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:45335 every 8 connection(s)
I20260929 15:53:25.313270 27570 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:53:25.320298 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 27570
I20260929 15:53:25.320538 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:53:25.321404 16147 external_mini_cluster.cc:1428] Setting key 4eb3a99baf9712d02b49c214eacee0eb
Sep 29 15:53:25 dist-test-slave-c51v krb5kdc[27014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697205, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:53:25.355774 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:25.316417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:50907 (local address 127.15.196.254:46497)
0929 15:53:25.329527 (+ 13110us) server_negotiation.cc:206] Beginning negotiation
0929 15:53:25.329535 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:53:25.334660 (+ 5125us) server_negotiation.cc:407] Connection header received
0929 15:53:25.334735 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:53:25.334740 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:53:25.334825 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:53:25.334935 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:53:25.337323 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:25.338522 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:25.341184 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:25.341846 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:25.341953 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:25.347447 (+ 5494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:53:25.347457 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:53:25.347464 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:53:25.347520 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:53:25.351473 (+ 3953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:53:25.352580 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:53:25.352589 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:53:25.352596 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:53:25.352767 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:53:25.353356 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:53:25.353364 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:53:25.353369 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:53:25.353751 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:53:25.353887 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 15:53:25.354958 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0929 15:53:25.355236 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12949,"thread_start_us":13114,"threads_started":1}
I20260929 15:53:25.358673 27706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:25.317095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:46497 (local address 127.15.196.193:50907)
0929 15:53:25.331527 (+ 14432us) negotiation.cc:107] Waiting for socket to connect
0929 15:53:25.331587 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 15:53:25.334144 (+ 2557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:53:25.335484 (+ 1340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:53:25.335498 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:53:25.336185 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:53:25.337082 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:53:25.337111 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:53:25.338755 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:53:25.338761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:53:25.340988 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:53:25.341003 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:53:25.342402 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:53:25.342411 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:53:25.342730 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:25.344006 (+ 1276us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:53:25.344051 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:53:25.347115 (+ 3064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:53:25.351669 (+ 4554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:53:25.351681 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:53:25.351705 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:53:25.352425 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:53:25.352944 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:53:25.352951 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:53:25.352957 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:53:25.353226 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:53:25.353981 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:53:25.353992 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:53:25.354495 (+ 503us) client_negotiation.cc:769] Sending connection context
0929 15:53:25.356414 (+ 1919us) client_negotiation.cc:240] Negotiation successful
0929 15:53:25.357046 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14023,"thread_start_us":22695,"threads_started":1}
I20260929 15:53:25.361083 27703 heartbeater.cc:347] Connected to a master server at 127.15.196.254:46497
I20260929 15:53:25.361462 27703 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
I20260929 15:53:25.362430 27703 heartbeater.cc:511] Master 127.15.196.254:46497 requested a full tablet report, sending...
I20260929 15:53:25.364846 27490 ts_manager.cc:194] Registered new tserver with Master: 4c1783cedc324c49a6d1f599e145c483 (127.15.196.193:45335)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
I20260929 15:53:25.366456 27490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:50907
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:53:25.399164 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.196.212:38791/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46497
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 15:53:25 dist-test-slave-c51v krb5kdc[27014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
Sep 29 15:53:25 dist-test-slave-c51v krb5kdc[27014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697204, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
W20260929 15:53:25.859611 27709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:53:25.860108 27709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:53:25.860296 27709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:53:25.872413 27709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:53:25.872664 27709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:53:25.872929 27709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:53:25.889107 27709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.196.212:38791/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46497
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:53:25.891268 27709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:53:25.893230 27709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:53:25.917109 27716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:53:25.918741 27717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:53:25.933997 27719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:53:25.935480 27709 server_base.cc:1034] running on GCE node
I20260929 15:53:25.936673 27709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:53:25.949955 27709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:53:25.951334 27709 hybrid_clock.cc:648] HybridClock initialized: now 1790697205951231 us; error 94 us; skew 500 ppm
I20260929 15:53:25.951831 27709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:53:25 dist-test-slave-c51v krb5kdc[27014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697205, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:53:25.956413 27709 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:53:25.986627 27709 webserver.cc:460] Webserver started at http://127.15.196.194:35389/ using document root <none> and password file <none>
I20260929 15:53:25.988004 27709 fs_manager.cc:359] Metadata directory not provided
I20260929 15:53:25.988283 27709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:53:25.988793 27709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 15:53:25 dist-test-slave-c51v krb5kdc[27014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697205, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
I20260929 15:53:26.025987 27709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "89a24f25c8c54227904cf93859f34f88"
format_stamp: "Formatted at 2026-09-29 15:53:26 on dist-test-slave-c51v"
server_key: "b8f882302e83b1efcaeb61334803a47e"
server_key_iv: "9eb2f78579667128b5d41157b3553e13"
server_key_version: "kuduclusterkey@0"
I20260929 15:53:26.027331 27709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "89a24f25c8c54227904cf93859f34f88"
format_stamp: "Formatted at 2026-09-29 15:53:26 on dist-test-slave-c51v"
server_key: "b8f882302e83b1efcaeb61334803a47e"
server_key_iv: "9eb2f78579667128b5d41157b3553e13"
server_key_version: "kuduclusterkey@0"
I20260929 15:53:26.041031 27709 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.007s sys 0.004s
I20260929 15:53:26.096292 27726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:53:26.098435 27709 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.006s sys 0.001s
I20260929 15:53:26.098743 27709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "89a24f25c8c54227904cf93859f34f88"
format_stamp: "Formatted at 2026-09-29 15:53:26 on dist-test-slave-c51v"
server_key: "b8f882302e83b1efcaeb61334803a47e"
server_key_iv: "9eb2f78579667128b5d41157b3553e13"
server_key_version: "kuduclusterkey@0"
I20260929 15:53:26.099081 27709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:53:26.158519 27709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:53:26.164320 27709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:53:26.164752 27709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:53:26.175750 27709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:53:26.187341 27709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:53:26.187515 27709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260929 15:53:26.187685 27709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:53:26.187788 27709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:53:26.372095 27703 heartbeater.cc:503] Master 127.15.196.254:46497 was elected leader, sending a full tablet report...
I20260929 15:53:26.622570 27709 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:37169
I20260929 15:53:26.622607 27845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:37169 every 8 connection(s)
I20260929 15:53:26.624733 27709 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:53:26.629249 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 27709
I20260929 15:53:26.629467 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:53:26.630132 16147 external_mini_cluster.cc:1428] Setting key 92d2a81a04a99bc5e0c14b1962298e54
Sep 29 15:53:26 dist-test-slave-c51v krb5kdc[27014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697205, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:53:26.653199 27848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:26.627762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:44411 (local address 127.15.196.254:46497)
0929 15:53:26.628316 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0929 15:53:26.628324 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:53:26.631003 (+ 2679us) server_negotiation.cc:407] Connection header received
0929 15:53:26.632291 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:53:26.632298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:53:26.632391 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:53:26.632514 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:53:26.634857 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:26.636053 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:26.638699 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:26.639462 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:26.639628 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:26.644546 (+ 4918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:53:26.644556 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:53:26.644564 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:53:26.644610 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:53:26.648642 (+ 4032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:53:26.649847 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:53:26.649856 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:53:26.649862 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:53:26.650019 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:53:26.650671 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:53:26.650679 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:53:26.650692 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:53:26.651073 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:53:26.651232 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 15:53:26.652406 (+ 1174us) server_negotiation.cc:299] Negotiation successful
0929 15:53:26.652673 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":254,"threads_started":1}
I20260929 15:53:26.654562 27849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:26.628475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:46497 (local address 127.15.196.194:44411)
0929 15:53:26.629291 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0929 15:53:26.629346 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 15:53:26.631801 (+ 2455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:53:26.632972 (+ 1171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:53:26.632986 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:53:26.633705 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:53:26.634611 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:53:26.634641 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:53:26.636249 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:53:26.636254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:53:26.638499 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:53:26.638517 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:53:26.639716 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:53:26.639726 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:53:26.640049 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:26.640995 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:53:26.641031 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:53:26.644258 (+ 3227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:53:26.648855 (+ 4597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:53:26.648868 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:53:26.648890 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:53:26.649672 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:53:26.650200 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:53:26.650206 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:53:26.650211 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:53:26.650521 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:53:26.651267 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:53:26.651277 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:53:26.651863 (+ 586us) client_negotiation.cc:769] Sending connection context
0929 15:53:26.652278 (+ 415us) client_negotiation.cc:240] Negotiation successful
0929 15:53:26.652905 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":359,"threads_started":1}
I20260929 15:53:26.656625 27846 heartbeater.cc:347] Connected to a master server at 127.15.196.254:46497
I20260929 15:53:26.657020 27846 heartbeater.cc:464] Registering TS with master...
I20260929 15:53:26.658106 27846 heartbeater.cc:511] Master 127.15.196.254:46497 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
I20260929 15:53:26.659955 27490 ts_manager.cc:194] Registered new tserver with Master: 89a24f25c8c54227904cf93859f34f88 (127.15.196.194:37169)
I20260929 15:53:26.660774 27490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:44411
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:53:26.691257 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.196.212:38791/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46497
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:53:27.028612 27852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:53:27.028903 27852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:53:27.029126 27852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:53:27.036739 27852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:53:27.036890 27852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:53:27.037087 27852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:53:27.047371 27852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.196.212:38791/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:46497
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:53:27.048746 27852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:53:27.050092 27852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:53:27.064227 27858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:53:27.064347 27857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:53:27.064397 27860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:53:27.064978 27852 server_base.cc:1034] running on GCE node
I20260929 15:53:27.065552 27852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:53:27.066747 27852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:53:27.067948 27852 hybrid_clock.cc:648] HybridClock initialized: now 1790697207067886 us; error 64 us; skew 500 ppm
I20260929 15:53:27.068234 27852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:53:27 dist-test-slave-c51v krb5kdc[27014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697207, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:53:27.071331 27852 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:53:27.072934 27852 webserver.cc:460] Webserver started at http://127.15.196.195:44703/ using document root <none> and password file <none>
I20260929 15:53:27.073618 27852 fs_manager.cc:359] Metadata directory not provided
I20260929 15:53:27.073752 27852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:53:27.074074 27852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 15:53:27 dist-test-slave-c51v krb5kdc[27014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697207, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
I20260929 15:53:27.089184 27852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "506a7382f95942249edd47015bcefd33"
format_stamp: "Formatted at 2026-09-29 15:53:27 on dist-test-slave-c51v"
server_key: "0cdd6f75c75002461b05c3a738faa2bd"
server_key_iv: "fb5136e0362a07b2e2ce22732cfb172b"
server_key_version: "kuduclusterkey@0"
I20260929 15:53:27.090063 27852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "506a7382f95942249edd47015bcefd33"
format_stamp: "Formatted at 2026-09-29 15:53:27 on dist-test-slave-c51v"
server_key: "0cdd6f75c75002461b05c3a738faa2bd"
server_key_iv: "fb5136e0362a07b2e2ce22732cfb172b"
server_key_version: "kuduclusterkey@0"
I20260929 15:53:27.096020 27852 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260929 15:53:27.111780 27867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:53:27.113488 27852 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260929 15:53:27.113692 27852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "506a7382f95942249edd47015bcefd33"
format_stamp: "Formatted at 2026-09-29 15:53:27 on dist-test-slave-c51v"
server_key: "0cdd6f75c75002461b05c3a738faa2bd"
server_key_iv: "fb5136e0362a07b2e2ce22732cfb172b"
server_key_version: "kuduclusterkey@0"
I20260929 15:53:27.113952 27852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:53:27.140905 27852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:53:27.145305 27852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:53:27.145565 27852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:53:27.146438 27852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:53:27.147814 27852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:53:27.147898 27852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:53:27.147979 27852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:53:27.148022 27852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:53:27.179833 27852 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:35277
I20260929 15:53:27.179868 27979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:35277 every 8 connection(s)
I20260929 15:53:27.181198 27852 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:53:27.185038 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 27852
I20260929 15:53:27.185221 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:53:27.185772 16147 external_mini_cluster.cc:1428] Setting key 26f7455fed7a286c312fe98d12d08897
Sep 29 15:53:27 dist-test-slave-c51v krb5kdc[27014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697207, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:53:27.202069 27982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:27.183846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:36275 (local address 127.15.196.254:46497)
0929 15:53:27.184403 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0929 15:53:27.184411 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:53:27.185613 (+ 1202us) server_negotiation.cc:407] Connection header received
0929 15:53:27.188083 (+ 2470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:53:27.188088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:53:27.188177 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:53:27.188293 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:53:27.189797 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.190522 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.192197 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.192604 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.192760 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:27.196222 (+ 3462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:53:27.196229 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:53:27.196234 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:53:27.196276 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:53:27.198665 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:53:27.199472 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:53:27.199477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:53:27.199480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:53:27.199639 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:53:27.200166 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:53:27.200170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:53:27.200174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:53:27.200571 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:53:27.200753 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0929 15:53:27.201618 (+ 865us) server_negotiation.cc:299] Negotiation successful
0929 15:53:27.201781 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":252,"threads_started":1}
I20260929 15:53:27.203083 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:27.184507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:46497 (local address 127.15.196.195:36275)
0929 15:53:27.185459 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0929 15:53:27.185499 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 15:53:27.187820 (+ 2321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:53:27.188543 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:53:27.188555 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:53:27.189058 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:53:27.189639 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:53:27.189657 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.190707 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.190712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:53:27.192045 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:53:27.192056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.192866 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.192874 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:53:27.193161 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:27.193901 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:53:27.193923 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:53:27.196007 (+ 2084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:53:27.198823 (+ 2816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:53:27.198832 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:53:27.198848 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:53:27.199322 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:53:27.199765 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:53:27.199769 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:53:27.199772 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:53:27.200035 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:53:27.200735 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:53:27.200742 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:53:27.201162 (+ 420us) client_negotiation.cc:769] Sending connection context
0929 15:53:27.201439 (+ 277us) client_negotiation.cc:240] Negotiation successful
0929 15:53:27.201921 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":393,"threads_started":1}
I20260929 15:53:27.204547 27980 heartbeater.cc:347] Connected to a master server at 127.15.196.254:46497
I20260929 15:53:27.204919 27980 heartbeater.cc:464] Registering TS with master...
I20260929 15:53:27.205634 27980 heartbeater.cc:511] Master 127.15.196.254:46497 requested a full tablet report, sending...
I20260929 15:53:27.207077 27490 ts_manager.cc:194] Registered new tserver with Master: 506a7382f95942249edd47015bcefd33 (127.15.196.195:35277)
I20260929 15:53:27.207935 27490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:36275
I20260929 15:53:27.213336 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:53:27.230810 27982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:27.217995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45842 (local address 127.15.196.254:46497)
0929 15:53:27.218155 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 15:53:27.218160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:53:27.218654 (+ 494us) server_negotiation.cc:407] Connection header received
0929 15:53:27.218825 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:53:27.218830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:53:27.218924 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:53:27.219008 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:53:27.220472 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.221206 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.222461 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.222877 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.222969 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:27.225529 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:53:27.225534 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:53:27.225538 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:53:27.225568 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:53:27.227939 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:53:27.228678 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:53:27.228683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:53:27.228687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:53:27.228904 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:53:27.229363 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:53:27.229371 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:53:27.229376 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:53:27.229805 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:53:27.230010 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0929 15:53:27.230328 (+ 318us) server_negotiation.cc:299] Negotiation successful
0929 15:53:27.230496 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 15:53:27.234727 27490 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:53:27.241771 27490 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:53:27.265877 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:27.257298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37208 (local address 127.15.196.193:45335)
0929 15:53:27.258581 (+ 1283us) server_negotiation.cc:206] Beginning negotiation
0929 15:53:27.258587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:53:27.258609 (+ 22us) server_negotiation.cc:407] Connection header received
0929 15:53:27.258711 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:53:27.258717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:53:27.258966 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:53:27.259165 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:53:27.259820 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.261194 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.263373 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.264690 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.264897 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:27.265084 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0929 15:53:27.265335 (+ 251us) server_negotiation.cc:299] Negotiation successful
0929 15:53:27.265515 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1130,"thread_start_us":982,"threads_started":1}
I20260929 15:53:27.265969 27997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:27.256985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:45335 (local address 127.0.0.1:37208)
0929 15:53:27.257834 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0929 15:53:27.257857 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 15:53:27.258006 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:53:27.259163 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:53:27.259168 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:53:27.259193 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:53:27.259620 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:53:27.259632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.261327 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.261333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:53:27.263139 (+ 1806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:53:27.263154 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.264897 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.264903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:53:27.265085 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:27.265142 (+ 57us) client_negotiation.cc:769] Sending connection context
0929 15:53:27.265292 (+ 150us) client_negotiation.cc:240] Negotiation successful
0929 15:53:27.265505 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":23,"thread_start_us":576,"threads_started":1}
I20260929 15:53:27.266776 27996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:27.255060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60956 (local address 127.15.196.195:35277)
0929 15:53:27.255892 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0929 15:53:27.255900 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:53:27.256559 (+ 659us) server_negotiation.cc:407] Connection header received
0929 15:53:27.256682 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:53:27.256692 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:53:27.257978 (+ 1286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:53:27.258165 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:53:27.259020 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.260627 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.263733 (+ 3106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.265193 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.265384 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:27.265521 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 15:53:27.266254 (+ 733us) server_negotiation.cc:299] Negotiation successful
0929 15:53:27.266434 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":474,"threads_started":1}
I20260929 15:53:27.266893 27994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:27.254858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48894 (local address 127.15.196.194:37169)
0929 15:53:27.255491 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0929 15:53:27.255503 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:53:27.257181 (+ 1678us) server_negotiation.cc:407] Connection header received
0929 15:53:27.257289 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:53:27.257299 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:53:27.257574 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:53:27.257761 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:53:27.258779 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.260446 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.263732 (+ 3286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.265193 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.265351 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:27.265521 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 15:53:27.266403 (+ 882us) server_negotiation.cc:299] Negotiation successful
0929 15:53:27.266568 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":301,"threads_started":1}
I20260929 15:53:27.267571 27995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:27.254847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:37169 (local address 127.0.0.1:48894)
0929 15:53:27.256911 (+ 2064us) negotiation.cc:107] Waiting for socket to connect
0929 15:53:27.256930 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:53:27.257049 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:53:27.258057 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:53:27.258068 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:53:27.258103 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:53:27.258547 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:53:27.258561 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.260560 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.260567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:53:27.263531 (+ 2964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:53:27.263545 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.265726 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.265733 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:53:27.265966 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:27.266083 (+ 117us) client_negotiation.cc:769] Sending connection context
0929 15:53:27.266151 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 15:53:27.267109 (+ 958us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1949,"mutex_wait_us":234,"thread_start_us":1808,"threads_started":1}
I20260929 15:53:27.267642 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:27.254480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:35277 (local address 127.0.0.1:60956)
0929 15:53:27.256094 (+ 1614us) negotiation.cc:107] Waiting for socket to connect
0929 15:53:27.256137 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 15:53:27.256338 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:53:27.258446 (+ 2108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:53:27.258452 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:53:27.258484 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:53:27.258838 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:53:27.258854 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.260779 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.260783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:53:27.263531 (+ 2748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:53:27.263592 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.265744 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.265750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:53:27.265963 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:27.266044 (+ 81us) client_negotiation.cc:769] Sending connection context
0929 15:53:27.266148 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 15:53:27.267089 (+ 941us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1335,"mutex_wait_us":939,"thread_start_us":1274,"threads_started":1}
I20260929 15:53:27.270479 27638 tablet_service.cc:1467] Processing CreateTablet for tablet cbfc5032064746ceb67fb28c987459d2 (DEFAULT_TABLE table=test-table [id=249bda2b38364cdba04bf8fc3bd75bba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:53:27.271816 27638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbfc5032064746ceb67fb28c987459d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:53:27.271335 27781 tablet_service.cc:1467] Processing CreateTablet for tablet cbfc5032064746ceb67fb28c987459d2 (DEFAULT_TABLE table=test-table [id=249bda2b38364cdba04bf8fc3bd75bba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:53:27.272739 27781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbfc5032064746ceb67fb28c987459d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:53:27.272300 27915 tablet_service.cc:1467] Processing CreateTablet for tablet cbfc5032064746ceb67fb28c987459d2 (DEFAULT_TABLE table=test-table [id=249bda2b38364cdba04bf8fc3bd75bba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:53:27.273643 27915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbfc5032064746ceb67fb28c987459d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:53:27.283464 27999 tablet_bootstrap.cc:492] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483: Bootstrap starting.
I20260929 15:53:27.286839 27999 tablet_bootstrap.cc:654] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483: Neither blocks nor log segments found. Creating new log.
I20260929 15:53:27.288261 27999 log.cc:824] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483: Log is configured to *not* fsync() on all Append() calls
I20260929 15:53:27.289430 28000 tablet_bootstrap.cc:492] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88: Bootstrap starting.
I20260929 15:53:27.292065 28001 tablet_bootstrap.cc:492] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33: Bootstrap starting.
I20260929 15:53:27.294203 28000 tablet_bootstrap.cc:654] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88: Neither blocks nor log segments found. Creating new log.
I20260929 15:53:27.295200 28001 tablet_bootstrap.cc:654] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33: Neither blocks nor log segments found. Creating new log.
I20260929 15:53:27.295778 28000 log.cc:824] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88: Log is configured to *not* fsync() on all Append() calls
I20260929 15:53:27.296411 28001 log.cc:824] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33: Log is configured to *not* fsync() on all Append() calls
I20260929 15:53:27.299183 28001 tablet_bootstrap.cc:492] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33: No bootstrap required, opened a new log
I20260929 15:53:27.299477 28001 ts_tablet_manager.cc:1397] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260929 15:53:27.299911 27999 tablet_bootstrap.cc:492] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483: No bootstrap required, opened a new log
I20260929 15:53:27.300256 27999 ts_tablet_manager.cc:1397] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.003s
I20260929 15:53:27.306469 28000 tablet_bootstrap.cc:492] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88: No bootstrap required, opened a new log
I20260929 15:53:27.306789 28000 ts_tablet_manager.cc:1397] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.008s
I20260929 15:53:27.307538 28001 raft_consensus.cc:348] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a7382f95942249edd47015bcefd33" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 35277 } } peers { permanent_uuid: "89a24f25c8c54227904cf93859f34f88" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37169 } } peers { permanent_uuid: "4c1783cedc324c49a6d1f599e145c483" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45335 } }
I20260929 15:53:27.308162 28001 raft_consensus.cc:374] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:53:27.308277 28001 raft_consensus.cc:729] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506a7382f95942249edd47015bcefd33, State: Initialized, Role: FOLLOWER
I20260929 15:53:27.308979 28001 consensus_queue.cc:260] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a7382f95942249edd47015bcefd33" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 35277 } } peers { permanent_uuid: "89a24f25c8c54227904cf93859f34f88" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37169 } } peers { permanent_uuid: "4c1783cedc324c49a6d1f599e145c483" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45335 } }
I20260929 15:53:27.310127 27980 heartbeater.cc:503] Master 127.15.196.254:46497 was elected leader, sending a full tablet report...
I20260929 15:53:27.310693 28001 ts_tablet_manager.cc:1428] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260929 15:53:27.312853 27999 raft_consensus.cc:348] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a7382f95942249edd47015bcefd33" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 35277 } } peers { permanent_uuid: "89a24f25c8c54227904cf93859f34f88" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37169 } } peers { permanent_uuid: "4c1783cedc324c49a6d1f599e145c483" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45335 } }
I20260929 15:53:27.313431 27999 raft_consensus.cc:374] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:53:27.313592 27999 raft_consensus.cc:729] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c1783cedc324c49a6d1f599e145c483, State: Initialized, Role: FOLLOWER
I20260929 15:53:27.314381 27999 consensus_queue.cc:260] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a7382f95942249edd47015bcefd33" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 35277 } } peers { permanent_uuid: "89a24f25c8c54227904cf93859f34f88" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37169 } } peers { permanent_uuid: "4c1783cedc324c49a6d1f599e145c483" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45335 } }
I20260929 15:53:27.321890 28000 raft_consensus.cc:348] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a7382f95942249edd47015bcefd33" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 35277 } } peers { permanent_uuid: "89a24f25c8c54227904cf93859f34f88" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37169 } } peers { permanent_uuid: "4c1783cedc324c49a6d1f599e145c483" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45335 } }
I20260929 15:53:27.322429 28000 raft_consensus.cc:374] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:53:27.322554 28000 raft_consensus.cc:729] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89a24f25c8c54227904cf93859f34f88, State: Initialized, Role: FOLLOWER
I20260929 15:53:27.323035 27999 ts_tablet_manager.cc:1428] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483: Time spent starting tablet: real 0.023s user 0.010s sys 0.011s
I20260929 15:53:27.323279 28000 consensus_queue.cc:260] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a7382f95942249edd47015bcefd33" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 35277 } } peers { permanent_uuid: "89a24f25c8c54227904cf93859f34f88" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37169 } } peers { permanent_uuid: "4c1783cedc324c49a6d1f599e145c483" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45335 } }
I20260929 15:53:27.324534 27846 heartbeater.cc:503] Master 127.15.196.254:46497 was elected leader, sending a full tablet report...
I20260929 15:53:27.325362 28000 ts_tablet_manager.cc:1428] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88: Time spent starting tablet: real 0.018s user 0.013s sys 0.005s
W20260929 15:53:27.333004 27704 tablet.cc:2369] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 15:53:27.379096 27847 tablet.cc:2369] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 15:53:27.432938 27981 tablet.cc:2369] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:53:27.658187 28006 raft_consensus.cc:482] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:53:27.658490 28006 raft_consensus.cc:504] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a7382f95942249edd47015bcefd33" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 35277 } } peers { permanent_uuid: "89a24f25c8c54227904cf93859f34f88" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37169 } } peers { permanent_uuid: "4c1783cedc324c49a6d1f599e145c483" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45335 } }
I20260929 15:53:27.660226 28006 leader_election.cc:288] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 506a7382f95942249edd47015bcefd33 (127.15.196.195:35277), 89a24f25c8c54227904cf93859f34f88 (127.15.196.194:37169)
I20260929 15:53:27.667264 27996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:27.660679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:40131 (local address 127.15.196.195:35277)
0929 15:53:27.660905 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 15:53:27.660914 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:53:27.661697 (+ 783us) server_negotiation.cc:407] Connection header received
0929 15:53:27.661842 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:53:27.661846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:53:27.661943 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:53:27.662038 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:53:27.662680 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.663458 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.665461 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.666306 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.666482 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:27.666560 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 15:53:27.666878 (+ 318us) server_negotiation.cc:299] Negotiation successful
0929 15:53:27.667006 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 15:53:27.667316 27994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:27.660748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:39941 (local address 127.15.196.194:37169)
0929 15:53:27.661105 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0929 15:53:27.661113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:53:27.661518 (+ 405us) server_negotiation.cc:407] Connection header received
0929 15:53:27.661724 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:53:27.661729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:53:27.661825 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:53:27.661902 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:53:27.662617 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.663423 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.665420 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.666260 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.666454 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:27.666534 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 15:53:27.666865 (+ 331us) server_negotiation.cc:299] Negotiation successful
0929 15:53:27.666991 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260929 15:53:27.667448 28008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:27.660506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:37169 (local address 127.15.196.193:39941)
0929 15:53:27.661423 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0929 15:53:27.661442 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:53:27.661565 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:53:27.661948 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:53:27.661953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:53:27.661978 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:53:27.662441 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:53:27.662458 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.663552 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.663560 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:53:27.665244 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:53:27.665257 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.666497 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.666505 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:53:27.666692 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:27.666725 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 15:53:27.666834 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 15:53:27.666968 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":795,"mutex_wait_us":24,"thread_start_us":521,"threads_started":1}
I20260929 15:53:27.667614 28009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:27.660674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:35277 (local address 127.15.196.193:40131)
0929 15:53:27.661558 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0929 15:53:27.661581 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 15:53:27.661722 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:53:27.662073 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:53:27.662076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:53:27.662107 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:53:27.662511 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:53:27.662541 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.663604 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.663607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:53:27.665216 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:53:27.665228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.666538 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.666544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:53:27.666715 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:27.666737 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 15:53:27.666854 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 15:53:27.666976 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"mutex_wait_us":67,"thread_start_us":430,"threads_started":1}
I20260929 15:53:27.668674 27935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbfc5032064746ceb67fb28c987459d2" candidate_uuid: "4c1783cedc324c49a6d1f599e145c483" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506a7382f95942249edd47015bcefd33" is_pre_election: true
I20260929 15:53:27.668669 27801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbfc5032064746ceb67fb28c987459d2" candidate_uuid: "4c1783cedc324c49a6d1f599e145c483" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89a24f25c8c54227904cf93859f34f88" is_pre_election: true
I20260929 15:53:27.669242 27935 raft_consensus.cc:2454] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c1783cedc324c49a6d1f599e145c483 in term 0.
I20260929 15:53:27.669245 27801 raft_consensus.cc:2454] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c1783cedc324c49a6d1f599e145c483 in term 0.
I20260929 15:53:27.669914 27592 leader_election.cc:302] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c1783cedc324c49a6d1f599e145c483, 506a7382f95942249edd47015bcefd33; no voters:
I20260929 15:53:27.670339 28006 raft_consensus.cc:2781] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:53:27.670487 28006 raft_consensus.cc:482] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:53:27.670590 28006 raft_consensus.cc:3037] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:53:27.672802 28006 raft_consensus.cc:504] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a7382f95942249edd47015bcefd33" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 35277 } } peers { permanent_uuid: "89a24f25c8c54227904cf93859f34f88" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37169 } } peers { permanent_uuid: "4c1783cedc324c49a6d1f599e145c483" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45335 } }
I20260929 15:53:27.673442 28006 leader_election.cc:288] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [CANDIDATE]: Term 1 election: Requested vote from peers 506a7382f95942249edd47015bcefd33 (127.15.196.195:35277), 89a24f25c8c54227904cf93859f34f88 (127.15.196.194:37169)
I20260929 15:53:27.673805 27935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbfc5032064746ceb67fb28c987459d2" candidate_uuid: "4c1783cedc324c49a6d1f599e145c483" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506a7382f95942249edd47015bcefd33"
I20260929 15:53:27.673913 27801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbfc5032064746ceb67fb28c987459d2" candidate_uuid: "4c1783cedc324c49a6d1f599e145c483" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89a24f25c8c54227904cf93859f34f88"
I20260929 15:53:27.674005 27935 raft_consensus.cc:3037] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:53:27.674158 27801 raft_consensus.cc:3037] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:53:27.676352 27935 raft_consensus.cc:2454] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c1783cedc324c49a6d1f599e145c483 in term 1.
I20260929 15:53:27.676410 27801 raft_consensus.cc:2454] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c1783cedc324c49a6d1f599e145c483 in term 1.
I20260929 15:53:27.676769 27592 leader_election.cc:302] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c1783cedc324c49a6d1f599e145c483, 506a7382f95942249edd47015bcefd33; no voters:
I20260929 15:53:27.677109 28006 raft_consensus.cc:2781] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:53:27.677539 28006 raft_consensus.cc:686] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [term 1 LEADER]: Becoming Leader. State: Replica: 4c1783cedc324c49a6d1f599e145c483, State: Running, Role: LEADER
I20260929 15:53:27.678053 28006 consensus_queue.cc:237] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a7382f95942249edd47015bcefd33" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 35277 } } peers { permanent_uuid: "89a24f25c8c54227904cf93859f34f88" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37169 } } peers { permanent_uuid: "4c1783cedc324c49a6d1f599e145c483" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45335 } }
I20260929 15:53:27.683109 27489 catalog_manager.cc:5495] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c1783cedc324c49a6d1f599e145c483 (127.15.196.193). New cstate: current_term: 1 leader_uuid: "4c1783cedc324c49a6d1f599e145c483" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a7382f95942249edd47015bcefd33" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 35277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89a24f25c8c54227904cf93859f34f88" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c1783cedc324c49a6d1f599e145c483" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 45335 } health_report { overall_health: HEALTHY } } }
I20260929 15:53:27.725193 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:27.718025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37214 (local address 127.15.196.193:45335)
0929 15:53:27.718259 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0929 15:53:27.718264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:53:27.718286 (+ 22us) server_negotiation.cc:407] Connection header received
0929 15:53:27.718557 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:53:27.718562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:53:27.718642 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:53:27.718812 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:53:27.719164 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.720004 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.721662 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.722339 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.722480 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:27.723169 (+ 689us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:53:27.723994 (+ 825us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:53:27.724165 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 15:53:27.724712 (+ 547us) server_negotiation.cc:299] Negotiation successful
0929 15:53:27.724829 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 15:53:27.734791 27801 raft_consensus.cc:1261] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88 [term 1 FOLLOWER]: Refusing update from remote peer 4c1783cedc324c49a6d1f599e145c483: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:53:27.735888 28010 consensus_queue.cc:1035] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89a24f25c8c54227904cf93859f34f88" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:53:27.736220 27935 raft_consensus.cc:1261] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33 [term 1 FOLLOWER]: Refusing update from remote peer 4c1783cedc324c49a6d1f599e145c483: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:53:27.740072 28010 consensus_queue.cc:1035] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [LEADER]: Connected to new peer: Peer: permanent_uuid: "506a7382f95942249edd47015bcefd33" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 35277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:53:27.763749 28014 mvcc.cc:205] Tried to move back new op lower bound from 7334695762872152064 to 7334695762653278208. Current Snapshot: MvccSnapshot[applied={T|T < 7334695762872152064}]
I20260929 15:53:27.780707 27490 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:53:27.781023 27490 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:53:27.785084 27490 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d9f361c1e2b466bada636bed10c9f03: Sending DeleteTablet for 3 replicas of tablet cbfc5032064746ceb67fb28c987459d2
I20260929 15:53:27.786356 27915 tablet_service.cc:1514] Processing DeleteTablet for tablet cbfc5032064746ceb67fb28c987459d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:53:27 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:60956
I20260929 15:53:27.786497 27781 tablet_service.cc:1514] Processing DeleteTablet for tablet cbfc5032064746ceb67fb28c987459d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:53:27 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:48894
I20260929 15:53:27.786633 27638 tablet_service.cc:1514] Processing DeleteTablet for tablet cbfc5032064746ceb67fb28c987459d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:53:27 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:37208
I20260929 15:53:27.789803 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 27570
I20260929 15:53:27.799865 28024 tablet_replica.cc:330] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88: stopping tablet replica
I20260929 15:53:27.799995 28025 tablet_replica.cc:330] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483: stopping tablet replica
I20260929 15:53:27.800428 28025 raft_consensus.cc:2229] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:53:27.800457 28024 raft_consensus.cc:2229] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:53:27.801095 28025 raft_consensus.cc:2258] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:53:27.801123 28024 raft_consensus.cc:2258] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:53:27.803936 28025 ts_tablet_manager.cc:1905] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:53:27.809358 28023 tablet_replica.cc:330] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33: stopping tablet replica
I20260929 15:53:27.809952 28023 raft_consensus.cc:2229] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:53:27.810653 28023 raft_consensus.cc:2258] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:53:27.812609 28025 ts_tablet_manager.cc:1918] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:53:27.812849 28025 log.cc:1196] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/cbfc5032064746ceb67fb28c987459d2
I20260929 15:53:27.813030 28024 ts_tablet_manager.cc:1905] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:53:27.813277 28023 ts_tablet_manager.cc:1905] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:53:27.813483 28025 ts_tablet_manager.cc:1939] T cbfc5032064746ceb67fb28c987459d2 P 4c1783cedc324c49a6d1f599e145c483: Deleting consensus metadata
I20260929 15:53:27.815454 27477 catalog_manager.cc:4841] TS 4c1783cedc324c49a6d1f599e145c483 (127.15.196.193:45335): tablet cbfc5032064746ceb67fb28c987459d2 (table test-table [id=249bda2b38364cdba04bf8fc3bd75bba]) successfully deleted
I20260929 15:53:27.820050 28023 ts_tablet_manager.cc:1918] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:53:27.820257 28024 ts_tablet_manager.cc:1918] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:53:27.820432 28024 log.cc:1196] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/cbfc5032064746ceb67fb28c987459d2
I20260929 15:53:27.820473 28023 log.cc:1196] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/cbfc5032064746ceb67fb28c987459d2
I20260929 15:53:27.821007 28024 ts_tablet_manager.cc:1939] T cbfc5032064746ceb67fb28c987459d2 P 89a24f25c8c54227904cf93859f34f88: Deleting consensus metadata
I20260929 15:53:27.821277 28023 ts_tablet_manager.cc:1939] T cbfc5032064746ceb67fb28c987459d2 P 506a7382f95942249edd47015bcefd33: Deleting consensus metadata
I20260929 15:53:27.822947 27477 catalog_manager.cc:4841] TS 89a24f25c8c54227904cf93859f34f88 (127.15.196.194:37169): tablet cbfc5032064746ceb67fb28c987459d2 (table test-table [id=249bda2b38364cdba04bf8fc3bd75bba]) successfully deleted
I20260929 15:53:27.823223 27475 catalog_manager.cc:4841] TS 506a7382f95942249edd47015bcefd33 (127.15.196.195:35277): tablet cbfc5032064746ceb67fb28c987459d2 (table test-table [id=249bda2b38364cdba04bf8fc3bd75bba]) successfully deleted
Sep 29 15:53:27 dist-test-slave-c51v krb5kdc[27014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:53:27.831739 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:27.792118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37226 (local address 127.15.196.193:45335)
0929 15:53:27.792399 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0929 15:53:27.792409 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:53:27.805934 (+ 13525us) server_negotiation.cc:407] Connection header received
0929 15:53:27.806496 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:53:27.806503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:53:27.806595 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:53:27.806687 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:53:27.809471 (+ 2784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.810811 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.812864 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.813504 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.813625 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:27.824802 (+ 11177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:53:27.824819 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:53:27.824834 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:53:27.824893 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:53:27.828365 (+ 3472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:53:27.829200 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:53:27.829209 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:53:27.829211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:53:27.829383 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:53:27.829846 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:53:27.829851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:53:27.829853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:53:27.830272 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:53:27.830607 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0929 15:53:27.831017 (+ 410us) server_negotiation.cc:299] Negotiation successful
0929 15:53:27.831318 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260929 15:53:27.832506 27698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:53:27.933676 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 27570
I20260929 15:53:27.952575 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 27709
Sep 29 15:53:27 dist-test-slave-c51v krb5kdc[27014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:53:27.967952 27994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:27.953374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48896 (local address 127.15.196.194:37169)
0929 15:53:27.953546 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 15:53:27.953553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:53:27.953643 (+ 90us) server_negotiation.cc:407] Connection header received
0929 15:53:27.953857 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:53:27.953863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:53:27.953926 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:53:27.954038 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:53:27.955320 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.956013 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.957295 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:27.957686 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:27.957824 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:27.962106 (+ 4282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:53:27.962120 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:53:27.962137 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:53:27.962186 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:53:27.965199 (+ 3013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:53:27.965938 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:53:27.965945 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:53:27.965947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:53:27.966069 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:53:27.966390 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:53:27.966393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:53:27.966395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:53:27.966725 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:53:27.966939 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0929 15:53:27.967178 (+ 239us) server_negotiation.cc:299] Negotiation successful
0929 15:53:27.967493 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 15:53:27.968585 27841 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:53:28.061147 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 27709
I20260929 15:53:28.078302 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 27852
Sep 29 15:53:28 dist-test-slave-c51v krb5kdc[27014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:53:28.094255 27996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:53:28.079097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60966 (local address 127.15.196.195:35277)
0929 15:53:28.079275 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 15:53:28.079281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:53:28.079354 (+ 73us) server_negotiation.cc:407] Connection header received
0929 15:53:28.079581 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:53:28.079585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:53:28.079658 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:53:28.079767 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:53:28.081092 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:28.081800 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:28.083066 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:53:28.083541 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:53:28.083672 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:53:28.088168 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:53:28.088183 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:53:28.088200 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:53:28.088234 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:53:28.090822 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:53:28.091674 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:53:28.091682 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:53:28.091684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:53:28.091908 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:53:28.092430 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:53:28.092435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:53:28.092439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:53:28.093011 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:53:28.093313 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0929 15:53:28.093660 (+ 347us) server_negotiation.cc:299] Negotiation successful
0929 15:53:28.093893 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 15:53:28.095110 27975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:53:28.185174 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 27852
I20260929 15:53:28.201614 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 27458
I20260929 15:53:28.202888 27520 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:53:28.316011 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 27458
I20260929 15:53:28.337450 16147 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 15:53:28.672636 16147 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 15:53:28.672822 16147 mini_ranger.cc:66] Stopping Ranger...
I20260929 15:53:29.018463 16147 mini_ranger.cc:68] Stopped Ranger
2026-09-29 15:53:29.018 UTC [27044] LOG: received smart shutdown request
2026-09-29 15:53:29.022 UTC [27044] LOG: background worker "logical replication launcher" (PID 27053) exited with exit code 1
2026-09-29 15:53:29.023 UTC [27048] LOG: shutting down
2026-09-29 15:53:29.206 UTC [27044] LOG: database system is shut down
2026-09-29T15:53:29Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51104 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:53:29 dist-test-slave-c51v krb5kdc[28039](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:53:29 dist-test-slave-c51v krb5kdc[28039](info): set up 2 sockets
Sep 29 15:53:29 dist-test-slave-c51v krb5kdc[28039](info): commencing operation
krb5kdc: starting...
W20260929 15:53:30.465579 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:53:30 dist-test-slave-c51v krb5kdc[28039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:53:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:53:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/rangeradmin_127.15.196.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/rangeradmin_127.15.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/rangerlookup_127.15.196.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/rangerlookup_127.15.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/HTTP_127.15.196.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/HTTP_127.15.196.212@KRBTEST.COM.keytab.
I20260929 15:53:30.794375 16147 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task6fGZba/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/postgres -l logfile start
2026-09-29 15:53:33.280 UTC [28069] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 15:53:33.280 UTC [28069] LOG: listening on IPv4 address "127.15.196.212", port 59723
2026-09-29 15:53:33.284 UTC [28069] LOG: listening on Unix socket "/tmp/.s.PGSQL.59723"
2026-09-29 15:53:33.307 UTC [28072] LOG: database system was shut down at 2026-09-29 15:53:31 UTC
2026-09-29 15:53:33.310 UTC [28069] LOG: database system is ready to accept connections
I20260929 15:53:34.293216 16147 mini_postgres.cc:96] Postgres bound to 59723
2026-09-29 15:53:34.310 UTC [28080] FATAL: database "slave" does not exist
127.15.196.212:59723 - accepting connections
I20260929 15:53:34.310896 16147 mini_ranger.cc:161] Starting Ranger...
I20260929 15:53:34.337365 16147 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 15:53:34.802845 16147 mini_ranger.cc:87] Created ranger Postgres database
I20260929 15:53:34.802968 16147 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-admin
2026-09-29 15:53:35,092 [I] DB FLAVOR :POSTGRES
2026-09-29 15:53:35,092 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 15:53:35,092 [I] Checking connection..
2026-09-29 15:53:35,092 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 15:53:35,353 [I] Checking connection passed.
2026-09-29 15:53:35,354 [I] --------- Verifying version history table ---------
2026-09-29 15:53:35,354 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 15:53:35,584 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 15:53:35,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 15:53:35,814 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 15:53:35,814 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 15:53:35,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 15:53:36.046 UTC [28144] WARNING: there is no transaction in progress
2026-09-29 15:53:36,053 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 15:53:36,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 15:53:36,311 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 15:53:36,311 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 15:53:36,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 15:53:36,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 15:53:36,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-c51v.c.gcp-upstream.internal','N') ;"
2026-09-29 15:53:37,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 15:53:37,292 [I] Table x_portal_user does not exist in database ranger
2026-09-29 15:53:37,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 15:53:37,547 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 15:53:37,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 15:53:37,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 15:53:38,031 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 15:53:38,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 15:53:38.956 UTC [28252] WARNING: there is no transaction in progress
2026-09-29 15:53:38.972 UTC [28252] WARNING: there is no transaction in progress
2026-09-29 15:53:38.988 UTC [28252] WARNING: there is no transaction in progress
2026-09-29 15:53:39.005 UTC [28252] WARNING: there is no transaction in progress
2026-09-29 15:53:39.024 UTC [28252] WARNING: there is no transaction in progress
2026-09-29 15:53:39.046 UTC [28252] WARNING: there is no transaction in progress
2026-09-29 15:53:39.114 UTC [28252] WARNING: there is no transaction in progress
2026-09-29 15:53:39.120 UTC [28252] WARNING: there is no transaction in progress
2026-09-29 15:53:39.135 UTC [28252] WARNING: there is no transaction in progress
2026-09-29 15:53:39.145 UTC [28252] WARNING: there is no transaction in progress
2026-09-29 15:53:39.156 UTC [28252] WARNING: there is no transaction in progress
2026-09-29 15:53:39.168 UTC [28252] WARNING: there is no transaction in progress
2026-09-29 15:53:39.178 UTC [28252] WARNING: there is no transaction in progress
2026-09-29 15:53:39.185 UTC [28252] WARNING: there is no transaction in progress
2026-09-29 15:53:39.192 UTC [28252] WARNING: there is no transaction in progress
2026-09-29 15:53:39.199 UTC [28252] WARNING: there is no transaction in progress
2026-09-29 15:53:39,824 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 15:53:39,825 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 15:53:40,066 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 15:53:40,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 15:53:40,305 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 15:53:40,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 15:53:40,537 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 15:53:40,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 15:53:40,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 15:53:41,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-c51v.c.gcp-upstream.internal';"
2026-09-29 15:53:41,297 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 15:53:41,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 15:53:41,523 [I] DB_PATCHES have already been applied
I20260929 15:53:41.529614 16147 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-admin:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task6fGZba/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 15:53:41.529695 16147 mini_ranger.cc:193] Using host: 127.15.196.212
I20260929 15:53:41.544186 16147 mini_ranger.cc:227] Ranger admin URL: http://127.15.196.212:39083
Sep 29, 2026 3:53:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 3:53:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 3:53:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 3:53:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 3:53:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 3:53:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 3:53:42 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 15:53:42.716454 16147 mini_ranger.cc:160] Time spent starting Ranger: real 8.406s user 0.005s sys 0.034s
Sep 29, 2026 3:54:02 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 3:54:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 3:54:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 3:54:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 3:54:02 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 3:54:03 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 15:54:04.308899 16147 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.196.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.196.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.196.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/rangerkms_127.15.196.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.196.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/rangerkms_127.15.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.196.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.196.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.196.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/HTTP_127.15.196.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.196.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/HTTP_127.15.196.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 15:54:04 dist-test-slave-c51v krb5kdc[28039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697244, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 15:54:04.467010 16147 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 15:54:04.486126 16147 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 15:54:05.340796 16147 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 15:54:05.340899 16147 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-kms
I20260929 15:54:05.340946 16147 mini_ranger_kms.cc:227] Using postgres at 127.15.196.212:59723
2026-09-29 15:54:05,704 [I] DB FLAVOR :POSTGRES
2026-09-29 15:54:05,705 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 15:54:05,705 [I] Checking connection
2026-09-29 15:54:05,705 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 15:54:05,966 [I] connection success
2026-09-29 15:54:05,966 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 15:54:05,966 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 15:54:05,966 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 15:54:06,204 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 15:54:06,204 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 15:54:06,204 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 15:54:06,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.196.212:59723/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 15:54:06,464 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 15:54:06.944430 16147 mini_ranger_kms.cc:310] Created kms service
I20260929 15:54:07.122601 16147 mini_ranger_kms.cc:326] Created kudu user
I20260929 15:54:07.166652 16147 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 15:54:07 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697244, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
I20260929 15:54:07.556756 16147 mini_ranger_kms.cc:384] Added ranger policy
I20260929 15:54:07.557102 16147 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-kms:/tmp/dist-test-task6fGZba/build/asan/bin/postgresql.jar:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task6fGZba/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 15:54:07.557164 16147 mini_ranger_kms.cc:240] Using host: 127.15.196.212
I20260929 15:54:07.574568 16147 mini_ranger_kms.cc:276] Ranger KMS PID: 28432
I20260929 15:54:07.574702 16147 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.196.212:51369
Sep 29, 2026 3:54:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 3:54:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 3:54:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 3:54:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 3:54:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 3:54:09 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 15:54:09.941960 16147 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.475s user 0.072s sys 0.303s
I20260929 15:54:09.942242 16147 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 15:54:09.942436 16147 mini_ranger_kms.cc:401] 127.15.196.212:51369/kms/v1/keys
Sep 29 15:54:10 dist-test-slave-c51v krb5kdc[28039](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790697250, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.196.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 15:54:10 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697250, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.196.212@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
Sep 29 15:54:10 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697250, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.196.212@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
Sep 29 15:54:13 dist-test-slave-c51v krb5kdc[28039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:54:13.174834 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:43589
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:41449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:43589
--ranger_config_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.196.212:51369/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:54:13.531245 28483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:13.531569 28483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:13.531662 28483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:13.538929 28483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:54:13.539050 28483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:13.539101 28483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:13.539165 28483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:54:13.539219 28483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:54:13.549576 28483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:41449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.196.212:51369/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:43589
--ranger_config_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:43589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:13.550810 28483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:13.552131 28483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:13.564308 28491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:13.564363 28489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:13.564599 28488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:13.565078 28483 server_base.cc:1034] running on GCE node
I20260929 15:54:13.565879 28483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:13.567360 28483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:13.568580 28483 hybrid_clock.cc:648] HybridClock initialized: now 1790697253568526 us; error 50 us; skew 500 ppm
I20260929 15:54:13.568928 28483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:13 dist-test-slave-c51v krb5kdc[28039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697253, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:13.571969 28483 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:54:13.573323 28483 webserver.cc:460] Webserver started at http://127.15.196.254:42153/ using document root <none> and password file <none>
I20260929 15:54:13.573926 28483 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:13.574029 28483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:13.574277 28483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 15:54:13 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697253, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
I20260929 15:54:13.631485 28483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "90e1706e1e8e46edae8948988270c04f"
format_stamp: "Formatted at 2026-09-29 15:54:13 on dist-test-slave-c51v"
server_key: "2954788be4f3d12197056371e9c880b4"
server_key_iv: "46fac90b41a592f59381efb8362097b9"
server_key_version: "kuduclusterkey@0"
I20260929 15:54:13.632388 28483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "90e1706e1e8e46edae8948988270c04f"
format_stamp: "Formatted at 2026-09-29 15:54:13 on dist-test-slave-c51v"
server_key: "2954788be4f3d12197056371e9c880b4"
server_key_iv: "46fac90b41a592f59381efb8362097b9"
server_key_version: "kuduclusterkey@0"
I20260929 15:54:13.638034 28483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260929 15:54:13.657936 28498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:13.659549 28483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:54:13.659741 28483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "90e1706e1e8e46edae8948988270c04f"
format_stamp: "Formatted at 2026-09-29 15:54:13 on dist-test-slave-c51v"
server_key: "2954788be4f3d12197056371e9c880b4"
server_key_iv: "46fac90b41a592f59381efb8362097b9"
server_key_version: "kuduclusterkey@0"
I20260929 15:54:13.659962 28483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:13.671443 28483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:13.675220 28483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:13.675580 28483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:13.694715 28483 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:43589
I20260929 15:54:13.694726 28549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:43589 every 8 connection(s)
I20260929 15:54:13.696157 28483 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:54:13.699849 28550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:13.701965 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 28483
I20260929 15:54:13.702193 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:54:13.702826 16147 external_mini_cluster.cc:1428] Setting key 037e52a1ced9fb0bbd2f495bc3e2aa9e
I20260929 15:54:13.711566 28550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f: Bootstrap starting.
I20260929 15:54:13.715189 28550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:13.716504 28550 log.cc:824] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f: Log is configured to *not* fsync() on all Append() calls
Sep 29 15:54:13 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:13.719328 28550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f: No bootstrap required, opened a new log
I20260929 15:54:13.725422 28553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:13.704932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39684 (local address 127.15.196.254:43589)
0929 15:54:13.705734 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:13.705746 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:13.705786 (+ 40us) server_negotiation.cc:407] Connection header received
0929 15:54:13.707463 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:13.707485 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:13.707859 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:13.708193 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:13.709837 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:13.711096 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:13.712927 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:13.713582 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:13.713714 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:13.718402 (+ 4688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:13.718410 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:13.718424 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:13.718459 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:13.721499 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:13.722143 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:13.722158 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:13.722163 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:13.722311 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:13.722695 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:13.722703 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:13.722708 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:13.723338 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:13.723664 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:13.724090 (+ 426us) server_negotiation.cc:299] Negotiation successful
0929 15:54:13.724477 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":391,"threads_started":1}
I20260929 15:54:13.726887 28550 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e1706e1e8e46edae8948988270c04f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43589 } }
I20260929 15:54:13.727315 28550 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:13.727442 28550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e1706e1e8e46edae8948988270c04f, State: Initialized, Role: FOLLOWER
I20260929 15:54:13.728052 28550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e1706e1e8e46edae8948988270c04f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43589 } }
I20260929 15:54:13.728322 28550 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:54:13.728453 28550 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:54:13.728592 28550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:13.730645 28550 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e1706e1e8e46edae8948988270c04f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43589 } }
I20260929 15:54:13.731065 28550 leader_election.cc:302] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90e1706e1e8e46edae8948988270c04f; no voters:
I20260929 15:54:13.731608 28550 leader_election.cc:288] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:54:13.731716 28555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:54:13.733358 28555 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 1 LEADER]: Becoming Leader. State: Replica: 90e1706e1e8e46edae8948988270c04f, State: Running, Role: LEADER
I20260929 15:54:13.733727 28550 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:54:13.734087 28555 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e1706e1e8e46edae8948988270c04f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43589 } }
I20260929 15:54:13.737828 28557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90e1706e1e8e46edae8948988270c04f. Latest consensus state: current_term: 1 leader_uuid: "90e1706e1e8e46edae8948988270c04f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e1706e1e8e46edae8948988270c04f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43589 } } }
I20260929 15:54:13.737993 28556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90e1706e1e8e46edae8948988270c04f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e1706e1e8e46edae8948988270c04f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43589 } } }
I20260929 15:54:13.738308 28557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:13.738387 28556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:13.740275 28561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:54:13.745939 28561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:54:13.746546 28550 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 15:54:13.770117 28561 catalog_manager.cc:1347] Generated new cluster ID: 0b1d411822b8486db320ca936b9679ae
I20260929 15:54:13.770323 28561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:54:13.790411 28561 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:54:13.792382 28561 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:54:13.799410 28561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f: Generated new TSK 0
I20260929 15:54:13.800523 28561 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:54:14.551635 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.196.212:51369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:43589
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:41449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 15:54:14 dist-test-slave-c51v krb5kdc[28039](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790697254, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 15:54:14.991921 28595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:14.992357 28595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:14.992591 28595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:15.005762 28595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:15.006052 28595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:15.006301 28595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:54:15.023916 28595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:41449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.196.212:51369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:43589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:15.025887 28595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:15.027783 28595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:15.049221 28605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:15.049258 28608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:15.049177 28606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:15.050482 28595 server_base.cc:1034] running on GCE node
I20260929 15:54:15.051203 28595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:15.052416 28595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:15.053666 28595 hybrid_clock.cc:648] HybridClock initialized: now 1790697255053607 us; error 51 us; skew 500 ppm
I20260929 15:54:15.054051 28595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:15 dist-test-slave-c51v krb5kdc[28039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697255, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:15.057986 28595 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:54:15.059749 28595 webserver.cc:460] Webserver started at http://127.15.196.193:36831/ using document root <none> and password file <none>
I20260929 15:54:15.060528 28595 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:15.060686 28595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:15.061022 28595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 15:54:15 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697255, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
I20260929 15:54:15.091969 28595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "bba41180083046dfbf4926c947fdd568"
format_stamp: "Formatted at 2026-09-29 15:54:15 on dist-test-slave-c51v"
server_key: "f7b8c1177a999068d5bd669e85318142"
server_key_iv: "ace04015881429f10ea360f25d057b0f"
server_key_version: "kuduclusterkey@0"
I20260929 15:54:15.093281 28595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "bba41180083046dfbf4926c947fdd568"
format_stamp: "Formatted at 2026-09-29 15:54:15 on dist-test-slave-c51v"
server_key: "f7b8c1177a999068d5bd669e85318142"
server_key_iv: "ace04015881429f10ea360f25d057b0f"
server_key_version: "kuduclusterkey@0"
I20260929 15:54:15.100649 28595 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260929 15:54:15.131997 28615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:15.134248 28595 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260929 15:54:15.134514 28595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "bba41180083046dfbf4926c947fdd568"
format_stamp: "Formatted at 2026-09-29 15:54:15 on dist-test-slave-c51v"
server_key: "f7b8c1177a999068d5bd669e85318142"
server_key_iv: "ace04015881429f10ea360f25d057b0f"
server_key_version: "kuduclusterkey@0"
I20260929 15:54:15.134860 28595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:15.173494 28595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:15.180058 28595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:15.180461 28595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:15.182160 28595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:15.188181 28595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:15.188293 28595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:15.188486 28595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:15.188580 28595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:15.229807 28595 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:38219
I20260929 15:54:15.229815 28727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:38219 every 8 connection(s)
I20260929 15:54:15.231863 28595 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:54:15.239997 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 28595
I20260929 15:54:15.240207 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:54:15.240906 16147 external_mini_cluster.cc:1428] Setting key dd92eb3d50b3ba42ff974cb4af1bab68
Sep 29 15:54:15 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697254, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
Sep 29 15:54:15 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697255, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:15.320734 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:15.242403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:55331 (local address 127.15.196.254:43589)
0929 15:54:15.279718 (+ 37315us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:15.279724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:15.279740 (+ 16us) server_negotiation.cc:407] Connection header received
0929 15:54:15.279792 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:15.279800 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:15.279906 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:15.280035 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:15.292534 (+ 12499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:15.294052 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:15.307090 (+ 13038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:15.307874 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:15.308003 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:15.312865 (+ 4862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:15.312873 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:15.312878 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:15.312922 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:15.316614 (+ 3692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:15.317716 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:15.317722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:15.317725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:15.317886 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:15.318496 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:15.318501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:15.318505 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:15.318857 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:15.318984 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:15.320066 (+ 1082us) server_negotiation.cc:299] Negotiation successful
0929 15:54:15.320289 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":10216,"server-negotiator.queue_time_us":37185,"thread_start_us":37047,"threads_started":1}
I20260929 15:54:15.323060 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:15.243046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:43589 (local address 127.15.196.193:55331)
0929 15:54:15.273453 (+ 30407us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:15.273512 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:15.276983 (+ 3471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:15.290471 (+ 13488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:15.290490 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:15.291313 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:15.292321 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:15.292352 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:15.297857 (+ 5505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:15.297863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:15.300018 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:15.300036 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:15.308451 (+ 8415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:15.308461 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:15.308820 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:15.309710 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:15.309750 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:15.312555 (+ 2805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:15.316909 (+ 4354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:15.316919 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:15.316939 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:15.317562 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:15.318047 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:15.318052 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:15.318055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:15.318341 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:15.319094 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:15.319102 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:15.319614 (+ 512us) client_negotiation.cc:769] Sending connection context
0929 15:54:15.321020 (+ 1406us) client_negotiation.cc:240] Negotiation successful
0929 15:54:15.321600 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30054,"thread_start_us":29788,"threads_started":1}
I20260929 15:54:15.325191 28728 heartbeater.cc:347] Connected to a master server at 127.15.196.254:43589
I20260929 15:54:15.325528 28728 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:15.326516 28728 heartbeater.cc:511] Master 127.15.196.254:43589 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
I20260929 15:54:15.329308 28515 ts_manager.cc:194] Registered new tserver with Master: bba41180083046dfbf4926c947fdd568 (127.15.196.193:38219)
I20260929 15:54:15.331233 28515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:55331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:54:15.373597 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.196.212:51369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:43589
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:41449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 15:54:15 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697254, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
W20260929 15:54:15.891228 28734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:15.891839 28734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:15.892076 28734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:15.904052 28734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:15.904280 28734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:15.904543 28734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:54:15.920643 28734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:41449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.196.212:51369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:43589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:15.922561 28734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:15.924468 28734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:15.942471 28741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:15.957830 28742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:15.958057 28744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:15.959666 28734 server_base.cc:1034] running on GCE node
I20260929 15:54:15.960552 28734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:15.961982 28734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:15.963261 28734 hybrid_clock.cc:648] HybridClock initialized: now 1790697255963145 us; error 102 us; skew 500 ppm
I20260929 15:54:15.963703 28734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:15 dist-test-slave-c51v krb5kdc[28039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697255, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:15.968261 28734 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:54:15.970191 28734 webserver.cc:460] Webserver started at http://127.15.196.194:40939/ using document root <none> and password file <none>
I20260929 15:54:15.971208 28734 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:15.971379 28734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:15.971755 28734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 15:54:15 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697255, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
I20260929 15:54:15.989521 28734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "b300f6abdc7541dbb273118123588aed"
format_stamp: "Formatted at 2026-09-29 15:54:15 on dist-test-slave-c51v"
server_key: "935028935bf21613bdbdac00423d288a"
server_key_iv: "72a32069db540f1944649e28d641fabf"
server_key_version: "kuduclusterkey@0"
I20260929 15:54:15.990777 28734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "b300f6abdc7541dbb273118123588aed"
format_stamp: "Formatted at 2026-09-29 15:54:15 on dist-test-slave-c51v"
server_key: "935028935bf21613bdbdac00423d288a"
server_key_iv: "72a32069db540f1944649e28d641fabf"
server_key_version: "kuduclusterkey@0"
I20260929 15:54:15.998531 28734 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260929 15:54:16.024930 28758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:16.027030 28734 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.001s
I20260929 15:54:16.027266 28734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "b300f6abdc7541dbb273118123588aed"
format_stamp: "Formatted at 2026-09-29 15:54:15 on dist-test-slave-c51v"
server_key: "935028935bf21613bdbdac00423d288a"
server_key_iv: "72a32069db540f1944649e28d641fabf"
server_key_version: "kuduclusterkey@0"
I20260929 15:54:16.027720 28734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:16.047102 28734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:16.052892 28734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:16.053239 28734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:16.054466 28734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:16.056275 28734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:16.056368 28734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:16.056555 28734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:16.056648 28734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:16.096392 28734 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:35709
I20260929 15:54:16.096438 28870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:35709 every 8 connection(s)
I20260929 15:54:16.098135 28734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:54:16.102851 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 28734
I20260929 15:54:16.103072 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:54:16.103824 16147 external_mini_cluster.cc:1428] Setting key b97a02b971d83c399797862a681702a0
Sep 29 15:54:16 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697255, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:16.130760 28874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:16.101923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:43589 (local address 127.15.196.194:58297)
0929 15:54:16.107896 (+ 5973us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:16.107946 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:16.110259 (+ 2313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:16.111664 (+ 1405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:16.111676 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:16.112262 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:16.113059 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:16.113084 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:16.114552 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:16.114558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:16.116510 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:16.116525 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:16.117521 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:16.117529 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:16.117826 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:16.118700 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:16.118730 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:16.121554 (+ 2824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:16.125983 (+ 4429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:16.125993 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:16.126013 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:16.126612 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:16.127056 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:16.127061 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:16.127065 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:16.127325 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:16.128029 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:16.128037 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:16.128487 (+ 450us) client_negotiation.cc:769] Sending connection context
0929 15:54:16.128782 (+ 295us) client_negotiation.cc:240] Negotiation successful
0929 15:54:16.129331 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5643,"thread_start_us":5399,"threads_started":1}
I20260929 15:54:16.132067 28873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:16.101134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:58297 (local address 127.15.196.254:43589)
0929 15:54:16.110946 (+ 9812us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:16.110952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:16.110972 (+ 20us) server_negotiation.cc:407] Connection header received
0929 15:54:16.111028 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:16.111033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:16.111111 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:16.111212 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:16.113260 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:16.114354 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:16.116698 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:16.117290 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:16.117434 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:16.121803 (+ 4369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:16.121810 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:16.121815 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:16.121856 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:16.125764 (+ 3908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:16.126762 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:16.126768 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:16.126772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:16.126928 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:16.127456 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:16.127461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:16.127465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:16.127810 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:16.127959 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:16.131458 (+ 3499us) server_negotiation.cc:299] Negotiation successful
0929 15:54:16.131713 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9666,"thread_start_us":18177,"threads_started":1}
I20260929 15:54:16.134312 28871 heartbeater.cc:347] Connected to a master server at 127.15.196.254:43589
I20260929 15:54:16.134668 28871 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:16.135550 28871 heartbeater.cc:511] Master 127.15.196.254:43589 requested a full tablet report, sending...
I20260929 15:54:16.137336 28515 ts_manager.cc:194] Registered new tserver with Master: b300f6abdc7541dbb273118123588aed (127.15.196.194:35709)
I20260929 15:54:16.138437 28515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:58297
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:54:16.179587 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.196.212:51369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:43589
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:41449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:54:16.335340 28728 heartbeater.cc:503] Master 127.15.196.254:43589 was elected leader, sending a full tablet report...
W20260929 15:54:16.587577 28877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:16.587880 28877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:16.587980 28877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:16.595495 28877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:16.595611 28877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:16.595727 28877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:54:16.605815 28877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:41449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.196.212:51369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:43589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:16.607091 28877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:16.608388 28877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:16.621829 28882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:16.621758 28885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:16.621791 28883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:16.622524 28877 server_base.cc:1034] running on GCE node
I20260929 15:54:16.623147 28877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:16.624099 28877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:16.625321 28877 hybrid_clock.cc:648] HybridClock initialized: now 1790697256625273 us; error 62 us; skew 500 ppm
I20260929 15:54:16.625634 28877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:16 dist-test-slave-c51v krb5kdc[28039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697256, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:16.628759 28877 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:54:16.630152 28877 webserver.cc:460] Webserver started at http://127.15.196.195:39405/ using document root <none> and password file <none>
I20260929 15:54:16.630877 28877 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:16.630973 28877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:16.631218 28877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 15:54:16 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697256, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
I20260929 15:54:16.648146 28877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "044a5493c6724406a4f09ed58fa4d758"
format_stamp: "Formatted at 2026-09-29 15:54:16 on dist-test-slave-c51v"
server_key: "332d2bb4ae94d0d568c21859a0b47e1f"
server_key_iv: "0b19ecd3e29979f7c6eb58841f5f694e"
server_key_version: "kuduclusterkey@0"
I20260929 15:54:16.649349 28877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "044a5493c6724406a4f09ed58fa4d758"
format_stamp: "Formatted at 2026-09-29 15:54:16 on dist-test-slave-c51v"
server_key: "332d2bb4ae94d0d568c21859a0b47e1f"
server_key_iv: "0b19ecd3e29979f7c6eb58841f5f694e"
server_key_version: "kuduclusterkey@0"
I20260929 15:54:16.655586 28877 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260929 15:54:16.670851 28892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:16.672477 28877 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260929 15:54:16.672657 28877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "044a5493c6724406a4f09ed58fa4d758"
format_stamp: "Formatted at 2026-09-29 15:54:16 on dist-test-slave-c51v"
server_key: "332d2bb4ae94d0d568c21859a0b47e1f"
server_key_iv: "0b19ecd3e29979f7c6eb58841f5f694e"
server_key_version: "kuduclusterkey@0"
I20260929 15:54:16.672875 28877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:16.684646 28877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:16.688623 28877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:16.688916 28877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:16.689891 28877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:16.691447 28877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:16.691519 28877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:16.691670 28877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:16.691726 28877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:16.724080 28877 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:32787
I20260929 15:54:16.724196 29004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:32787 every 8 connection(s)
I20260929 15:54:16.725358 28877 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:54:16.734545 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 28877
I20260929 15:54:16.734740 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:54:16.735350 16147 external_mini_cluster.cc:1428] Setting key 1907019e84befaff42e832738a9e5435
Sep 29 15:54:16 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697256, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:16.745283 29007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:16.727893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:41207 (local address 127.15.196.254:43589)
0929 15:54:16.728375 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:16.728380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:16.729286 (+ 906us) server_negotiation.cc:407] Connection header received
0929 15:54:16.731072 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:16.731077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:16.731163 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:16.731294 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:16.732696 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:16.733475 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:16.735370 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:16.735835 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:16.735965 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:16.739387 (+ 3422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:16.739392 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:16.739395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:16.739425 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:16.742083 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:16.742772 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:16.742779 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:16.742782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:16.742907 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:16.743339 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:16.743345 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:16.743348 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:16.743695 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:16.743861 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:16.744670 (+ 809us) server_negotiation.cc:299] Negotiation successful
0929 15:54:16.744931 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":239,"threads_started":1}
I20260929 15:54:16.746392 29008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:16.728464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:43589 (local address 127.15.196.195:41207)
0929 15:54:16.729120 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:16.729163 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:16.730775 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:16.731558 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:16.731566 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:16.731971 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:16.732533 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:16.732549 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:16.733643 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:16.733652 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:16.735197 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:16.735208 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:16.736050 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:16.736057 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:16.736307 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:16.736983 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:16.737008 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:16.739159 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:16.742224 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:16.742232 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:16.742248 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:16.742652 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:16.743019 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:16.743026 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:16.743030 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:16.743210 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:16.743852 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:16.743859 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:16.744290 (+ 431us) client_negotiation.cc:769] Sending connection context
0929 15:54:16.744580 (+ 290us) client_negotiation.cc:240] Negotiation successful
0929 15:54:16.745070 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":255,"threads_started":1}
I20260929 15:54:16.747985 29005 heartbeater.cc:347] Connected to a master server at 127.15.196.254:43589
I20260929 15:54:16.748296 29005 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:16.749033 29005 heartbeater.cc:511] Master 127.15.196.254:43589 requested a full tablet report, sending...
I20260929 15:54:16.750629 28515 ts_manager.cc:194] Registered new tserver with Master: 044a5493c6724406a4f09ed58fa4d758 (127.15.196.195:32787)
I20260929 15:54:16.751523 28515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:41207
I20260929 15:54:16.762616 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:54:16.764356 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 28595
Sep 29 15:54:16 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:54:16.780951 29010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:16.765355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43912 (local address 127.15.196.193:38219)
0929 15:54:16.766030 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:16.766036 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:16.766060 (+ 24us) server_negotiation.cc:407] Connection header received
0929 15:54:16.766115 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:16.766122 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:16.766330 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:16.766450 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:16.767935 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:16.768916 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:16.770312 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:16.770854 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:16.770950 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:16.775268 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:16.775279 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:16.775336 (+ 57us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:16.775377 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:16.778036 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:16.778768 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:16.778775 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:16.778778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:16.778918 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:16.779362 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:16.779371 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:16.779373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:16.779687 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:16.780010 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:16.780278 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 15:54:16.780541 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":360,"threads_started":1}
I20260929 15:54:16.782184 28723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:16.865501 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 28595
I20260929 15:54:16.881697 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 28734
Sep 29 15:54:16 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:54:16.897425 29012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:16.882590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33990 (local address 127.15.196.194:35709)
0929 15:54:16.883156 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:16.883163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:16.883188 (+ 25us) server_negotiation.cc:407] Connection header received
0929 15:54:16.883248 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:16.883254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:16.883553 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:16.883681 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:16.884872 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:16.885877 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:16.887360 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:16.887917 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:16.888013 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:16.892244 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:16.892255 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:16.892269 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:16.892301 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:16.895033 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:16.895765 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:16.895773 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:16.895775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:16.895905 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:16.896198 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:16.896201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:16.896203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:16.896505 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:16.896735 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:16.896936 (+ 201us) server_negotiation.cc:299] Negotiation successful
0929 15:54:16.897141 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":297,"threads_started":1}
I20260929 15:54:16.898831 28866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:16.987068 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 28734
I20260929 15:54:17.003248 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 28877
Sep 29 15:54:17 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:54:17.019654 29014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:17.004163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53330 (local address 127.15.196.195:32787)
0929 15:54:17.004719 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:17.004726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:17.004753 (+ 27us) server_negotiation.cc:407] Connection header received
0929 15:54:17.004811 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:17.004817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:17.005036 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:17.005219 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:17.006533 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:17.007612 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:17.008883 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:17.009452 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:17.009584 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:17.013914 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:17.013925 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:17.013940 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:17.013971 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:17.016751 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:17.017427 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:17.017434 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:17.017436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:17.017561 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:17.017950 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:17.017956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:17.017959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:17.018299 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:17.018639 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:17.018947 (+ 308us) server_negotiation.cc:299] Negotiation successful
0929 15:54:17.019220 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":260,"threads_started":1}
I20260929 15:54:17.021116 29000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:17.107188 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 28877
I20260929 15:54:17.123190 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 28483
I20260929 15:54:17.124658 28545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:17.231945 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 28483
I20260929 15:54:17.253715 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:43589
--webserver_interface=127.15.196.254
--webserver_port=42153
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:41449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:43589
--ranger_config_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.196.212:51369/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.196.212:51369/kms with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:54:17.574168 29017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:17.574476 29017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:17.574632 29017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:17.582001 29017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:54:17.582208 29017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:17.582307 29017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:17.582408 29017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:54:17.582492 29017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:54:17.592746 29017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:41449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.196.212:51369/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:43589
--ranger_config_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:43589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=42153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:17.594039 29017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:17.595438 29017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:17.606980 29023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:17.607060 29022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:17.607031 29025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:17.607889 29017 server_base.cc:1034] running on GCE node
I20260929 15:54:17.608696 29017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:17.610172 29017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:17.611382 29017 hybrid_clock.cc:648] HybridClock initialized: now 1790697257611326 us; error 52 us; skew 500 ppm
I20260929 15:54:17.611720 29017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:17 dist-test-slave-c51v krb5kdc[28039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697257, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:17.618723 29017 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:54:17.620286 29017 webserver.cc:460] Webserver started at http://127.15.196.254:42153/ using document root <none> and password file <none>
I20260929 15:54:17.620965 29017 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:17.621114 29017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 15:54:17 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697257, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
I20260929 15:54:17.670869 29017 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260929 15:54:17.673816 29033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:17.675252 29017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:54:17.675516 29017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "90e1706e1e8e46edae8948988270c04f"
format_stamp: "Formatted at 2026-09-29 15:54:13 on dist-test-slave-c51v"
server_key: "2954788be4f3d12197056371e9c880b4"
server_key_iv: "46fac90b41a592f59381efb8362097b9"
server_key_version: "kuduclusterkey@0"
I20260929 15:54:17.676540 29017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:17.701344 29017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:17.705109 29017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:17.705340 29017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:17.724602 29017 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:43589
I20260929 15:54:17.724641 29084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:43589 every 8 connection(s)
I20260929 15:54:17.726261 29017 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:54:17.733031 29085 sys_catalog.cc:263] Verifying existing consensus state
I20260929 15:54:17.735337 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 29017
I20260929 15:54:17.736387 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.196.212:51369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:38219
--local_ip_for_outbound_sockets=127.15.196.193
--tserver_master_addrs=127.15.196.254:43589
--webserver_port=36831
--webserver_interface=127.15.196.193
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:41449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.196.212:51369/kms with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:54:17.736970 29085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f: Bootstrap starting.
I20260929 15:54:17.759168 29085 log.cc:824] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:17.768075 29085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 15:54:17.768616 29085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f: Bootstrap complete.
I20260929 15:54:17.777726 29085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e1706e1e8e46edae8948988270c04f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43589 } }
I20260929 15:54:17.778541 29085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e1706e1e8e46edae8948988270c04f, State: Initialized, Role: FOLLOWER
I20260929 15:54:17.779163 29085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e1706e1e8e46edae8948988270c04f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43589 } }
I20260929 15:54:17.779414 29085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:54:17.779572 29085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:54:17.779791 29085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 1 FOLLOWER]: Advancing to term 2
I20260929 15:54:17.783684 29085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e1706e1e8e46edae8948988270c04f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43589 } }
I20260929 15:54:17.784170 29085 leader_election.cc:302] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90e1706e1e8e46edae8948988270c04f; no voters:
I20260929 15:54:17.784883 29085 leader_election.cc:288] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 15:54:17.785032 29089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 2 FOLLOWER]: Leader election won for term 2
I20260929 15:54:17.787220 29085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:54:17.792044 29089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [term 2 LEADER]: Becoming Leader. State: Replica: 90e1706e1e8e46edae8948988270c04f, State: Running, Role: LEADER
I20260929 15:54:17.792729 29089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e1706e1e8e46edae8948988270c04f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43589 } }
I20260929 15:54:17.800385 29091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90e1706e1e8e46edae8948988270c04f. Latest consensus state: current_term: 2 leader_uuid: "90e1706e1e8e46edae8948988270c04f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e1706e1e8e46edae8948988270c04f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43589 } } }
I20260929 15:54:17.800865 29091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:17.801524 29090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "90e1706e1e8e46edae8948988270c04f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e1706e1e8e46edae8948988270c04f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43589 } } }
I20260929 15:54:17.801842 29090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:17.805393 29104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:54:17.806339 29085 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 15:54:17.811789 29104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:54:17.813520 29104 catalog_manager.cc:1259] Loaded cluster ID: 0b1d411822b8486db320ca936b9679ae
I20260929 15:54:17.813635 29104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:54:17.827194 29104 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:54:17.831881 29104 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f: Loaded TSK: 0
I20260929 15:54:17.833599 29104 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 15:54:18.203642 29087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:18.204087 29087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:18.204300 29087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:18.216791 29087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:18.216936 29087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:18.217133 29087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:54:18.233104 29087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:41449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.196.212:51369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:38219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=36831
--tserver_master_addrs=127.15.196.254:43589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:18.234869 29087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:18.236801 29087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:18.260319 29123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:18.260304 29124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:18.269313 29087 server_base.cc:1034] running on GCE node
W20260929 15:54:18.269714 29126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:18.270610 29087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:18.272922 29087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:18.276284 29087 hybrid_clock.cc:648] HybridClock initialized: now 1790697258276219 us; error 51 us; skew 500 ppm
I20260929 15:54:18.276778 29087 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:18 dist-test-slave-c51v krb5kdc[28039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697258, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:18.281514 29087 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:54:18.285008 29087 webserver.cc:460] Webserver started at http://127.15.196.193:36831/ using document root <none> and password file <none>
I20260929 15:54:18.285920 29087 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:18.286065 29087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 15:54:18 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697258, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
I20260929 15:54:18.325121 29087 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.001s
I20260929 15:54:18.338349 29134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:18.341146 29087 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.006s sys 0.007s
I20260929 15:54:18.341408 29087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "bba41180083046dfbf4926c947fdd568"
format_stamp: "Formatted at 2026-09-29 15:54:15 on dist-test-slave-c51v"
server_key: "f7b8c1177a999068d5bd669e85318142"
server_key_iv: "ace04015881429f10ea360f25d057b0f"
server_key_version: "kuduclusterkey@0"
I20260929 15:54:18.342718 29087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:18.379439 29087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:18.385178 29087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:18.385528 29087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:18.387565 29087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:18.389614 29087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:18.389753 29087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:18.389885 29087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:18.389978 29087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:18.469848 29087 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:38219
I20260929 15:54:18.469859 29247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:38219 every 8 connection(s)
I20260929 15:54:18.472008 29087 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:54:18.478797 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 29087
I20260929 15:54:18.479713 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.196.212:51369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:35709
--local_ip_for_outbound_sockets=127.15.196.194
--tserver_master_addrs=127.15.196.254:43589
--webserver_port=40939
--webserver_interface=127.15.196.194
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:41449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.196.212:51369/kms with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 15:54:18 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697258, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:18.504621 29250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:18.475492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:44619 (local address 127.15.196.254:43589)
0929 15:54:18.476321 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:18.476359 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:18.476956 (+ 597us) server_negotiation.cc:407] Connection header received
0929 15:54:18.480073 (+ 3117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:18.480091 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:18.480625 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:18.481036 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:18.482564 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:18.483873 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:18.485877 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:18.486518 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:18.486723 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:18.492903 (+ 6180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:18.492917 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:18.492942 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:18.493001 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:18.497699 (+ 4698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:18.499113 (+ 1414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:18.499125 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:18.499135 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:18.499464 (+ 329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:18.500125 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:18.500132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:18.500137 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:18.500927 (+ 790us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:18.501137 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:18.502465 (+ 1328us) server_negotiation.cc:299] Negotiation successful
0929 15:54:18.502982 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":299,"threads_started":1}
I20260929 15:54:18.507097 29251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:18.475590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:43589 (local address 127.15.196.193:44619)
0929 15:54:18.476750 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:18.476810 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:18.479340 (+ 2530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:18.481237 (+ 1897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:18.481248 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:18.481784 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:18.482347 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:18.482365 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:18.484036 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:18.484042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:18.485714 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:18.485726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:18.486768 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:18.486775 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:18.487096 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:18.487986 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:18.488025 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:18.492528 (+ 4503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:18.498089 (+ 5561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:18.498102 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:18.498123 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:18.498889 (+ 766us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:18.499685 (+ 796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:18.499691 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:18.499695 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:18.499965 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:18.501266 (+ 1301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:18.501276 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:18.501832 (+ 556us) client_negotiation.cc:769] Sending connection context
0929 15:54:18.505058 (+ 3226us) client_negotiation.cc:240] Negotiation successful
0929 15:54:18.505704 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"thread_start_us":710,"threads_started":1}
I20260929 15:54:18.511415 29248 heartbeater.cc:347] Connected to a master server at 127.15.196.254:43589
I20260929 15:54:18.511811 29248 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:18.512799 29248 heartbeater.cc:511] Master 127.15.196.254:43589 requested a full tablet report, sending...
W20260929 15:54:18.516247 29248 heartbeater.cc:644] Failed to heartbeat to 127.15.196.254:43589 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 15:54:18.516523 29248 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:18.516860 29248 heartbeater.cc:511] Master 127.15.196.254:43589 requested a full tablet report, sending...
I20260929 15:54:18.518158 29248 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:18.518464 29248 heartbeater.cc:511] Master 127.15.196.254:43589 requested a full tablet report, sending...
W20260929 15:54:18.520094 29248 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 15:54:18.908095 29252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:18.908354 29252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:18.908510 29252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:18.917989 29252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:18.918200 29252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:18.918404 29252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:54:18.933895 29252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:41449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.196.212:51369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:35709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=40939
--tserver_master_addrs=127.15.196.254:43589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:18.935611 29252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:18.937258 29252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:18.960125 29266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:18.961292 29267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:18.966405 29252 server_base.cc:1034] running on GCE node
W20260929 15:54:18.966675 29269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:18.967406 29252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:18.968554 29252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:18.969777 29252 hybrid_clock.cc:648] HybridClock initialized: now 1790697258969688 us; error 82 us; skew 500 ppm
I20260929 15:54:18.970129 29252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:18 dist-test-slave-c51v krb5kdc[28039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697258, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:18.973613 29252 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:54:18.976799 29252 webserver.cc:460] Webserver started at http://127.15.196.194:40939/ using document root <none> and password file <none>
I20260929 15:54:18.977566 29252 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:18.977731 29252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 15:54:18 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697258, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
I20260929 15:54:19.015413 29252 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260929 15:54:19.019371 29279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:19.021277 29252 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 15:54:19.021543 29252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "b300f6abdc7541dbb273118123588aed"
format_stamp: "Formatted at 2026-09-29 15:54:15 on dist-test-slave-c51v"
server_key: "935028935bf21613bdbdac00423d288a"
server_key_iv: "72a32069db540f1944649e28d641fabf"
server_key_version: "kuduclusterkey@0"
I20260929 15:54:19.023113 29252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:19.057446 29252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:19.062435 29252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:19.062764 29252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:19.064059 29252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:19.065743 29252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:19.065872 29252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:19.066026 29252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:19.066123 29252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 15:54:19 dist-test-slave-c51v krb5kdc[28039](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790697259, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:19.150418 29252 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:35709
I20260929 15:54:19.150520 29391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:35709 every 8 connection(s)
I20260929 15:54:19.152290 29252 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:54:19.162213 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 29252
I20260929 15:54:19.162992 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.196.212:51369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:32787
--local_ip_for_outbound_sockets=127.15.196.195
--tserver_master_addrs=127.15.196.254:43589
--webserver_port=39405
--webserver_interface=127.15.196.195
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:41449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.196.212:51369/kms with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 15:54:19 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697258, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:19.214257 29394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:19.155725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:43589 (local address 127.15.196.194:34645)
0929 15:54:19.190264 (+ 34539us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:19.190326 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:19.194317 (+ 3991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:19.195920 (+ 1603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:19.195933 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:19.196641 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:19.197421 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:19.197439 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:19.198583 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:19.198589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:19.199958 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:19.199977 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:19.200800 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:19.200809 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:19.201112 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:19.201945 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:19.201983 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:19.204859 (+ 2876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:19.209284 (+ 4425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:19.209293 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:19.209313 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:19.209904 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:19.210360 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:19.210376 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:19.210380 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:19.210606 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:19.211408 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:19.211417 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:19.211872 (+ 455us) client_negotiation.cc:769] Sending connection context
0929 15:54:19.212190 (+ 318us) client_negotiation.cc:240] Negotiation successful
0929 15:54:19.212811 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34171,"thread_start_us":37051,"threads_started":1}
I20260929 15:54:19.215381 29395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:19.156315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:34645 (local address 127.15.196.254:43589)
0929 15:54:19.189516 (+ 33201us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:19.189523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:19.192863 (+ 3340us) server_negotiation.cc:407] Connection header received
0929 15:54:19.195106 (+ 2243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:19.195113 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:19.195211 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:19.195327 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:19.197621 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:19.198412 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:19.200119 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:19.200556 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:19.200696 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:19.205434 (+ 4738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:19.205441 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:19.205446 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:19.205487 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:19.209118 (+ 3631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:19.210058 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:19.210065 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:19.210068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:19.210224 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:19.210754 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:19.210767 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:19.210770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:19.211125 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:19.211315 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:19.214766 (+ 3451us) server_negotiation.cc:299] Negotiation successful
0929 15:54:19.215001 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33034,"thread_start_us":33222,"threads_started":1}
I20260929 15:54:19.217638 29392 heartbeater.cc:347] Connected to a master server at 127.15.196.254:43589
I20260929 15:54:19.217960 29392 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:19.218832 29392 heartbeater.cc:511] Master 127.15.196.254:43589 requested a full tablet report, sending...
I20260929 15:54:19.220963 29050 ts_manager.cc:194] Registered new tserver with Master: b300f6abdc7541dbb273118123588aed (127.15.196.194:35709)
I20260929 15:54:19.223140 29050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:34645
I20260929 15:54:19.520260 29248 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:19.520732 29248 heartbeater.cc:511] Master 127.15.196.254:43589 requested a full tablet report, sending...
I20260929 15:54:19.522148 29050 ts_manager.cc:194] Registered new tserver with Master: bba41180083046dfbf4926c947fdd568 (127.15.196.193:38219)
I20260929 15:54:19.523459 29050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:44619
Sep 29 15:54:19 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697259, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
W20260929 15:54:19.669555 29396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:19.670050 29396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:19.670248 29396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:19.683483 29396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:19.683925 29396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:19.684201 29396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:54:19.705406 29396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:41449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.196.212:51369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:32787
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=39405
--tserver_master_addrs=127.15.196.254:43589
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:19.707356 29396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:19.709219 29396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:19.729300 29402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:19.730940 29403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:19.737200 29406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:19.737728 29396 server_base.cc:1034] running on GCE node
I20260929 15:54:19.738667 29396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:19.739972 29396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:19.741235 29396 hybrid_clock.cc:648] HybridClock initialized: now 1790697259741162 us; error 65 us; skew 500 ppm
I20260929 15:54:19.741653 29396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:19 dist-test-slave-c51v krb5kdc[28039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697259, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:19.745102 29396 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:54:19.748705 29396 webserver.cc:460] Webserver started at http://127.15.196.195:39405/ using document root <none> and password file <none>
I20260929 15:54:19.749516 29396 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:19.749681 29396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 15:54:19 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697259, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
I20260929 15:54:19.778469 29396 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260929 15:54:19.782733 29413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:19.784837 29396 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260929 15:54:19.785058 29396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "044a5493c6724406a4f09ed58fa4d758"
format_stamp: "Formatted at 2026-09-29 15:54:16 on dist-test-slave-c51v"
server_key: "332d2bb4ae94d0d568c21859a0b47e1f"
server_key_iv: "0b19ecd3e29979f7c6eb58841f5f694e"
server_key_version: "kuduclusterkey@0"
I20260929 15:54:19.786679 29396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:19.821081 29396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:19.826599 29396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:19.826910 29396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:19.834544 29396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:19.842710 29396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:19.842845 29396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:19.842970 29396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:19.843056 29396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 15:54:20 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697259, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for HTTP/127.15.196.212@KRBTEST.COM
I20260929 15:54:20.227759 29392 heartbeater.cc:503] Master 127.15.196.254:43589 was elected leader, sending a full tablet report...
I20260929 15:54:20.238530 29396 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:32787
I20260929 15:54:20.238619 29526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:32787 every 8 connection(s)
I20260929 15:54:20.240324 29396 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:54:20.245848 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 29396
I20260929 15:54:20.305325 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.279818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54342 (local address 127.15.196.254:43589)
0929 15:54:20.283484 (+ 3666us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:20.283491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:20.283508 (+ 17us) server_negotiation.cc:407] Connection header received
0929 15:54:20.283574 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:20.283580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:20.283687 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:20.283910 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:20.285967 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.287191 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.289201 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.289934 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.290068 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.293694 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:20.293704 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:20.293711 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:20.293764 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:20.297838 (+ 4074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:20.298782 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:20.298788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:20.298792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:20.298964 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:20.303023 (+ 4059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:20.303028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:20.303032 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:20.303441 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:20.303720 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:20.304348 (+ 628us) server_negotiation.cc:299] Negotiation successful
0929 15:54:20.304930 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3525,"thread_start_us":3308,"threads_started":1}
Sep 29 15:54:20 dist-test-slave-c51v krb5kdc[28039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697259, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:20.317582 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.244147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:35447 (local address 127.15.196.254:43589)
0929 15:54:20.279767 (+ 35620us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:20.279775 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:20.283316 (+ 3541us) server_negotiation.cc:407] Connection header received
0929 15:54:20.286016 (+ 2700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:20.286023 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:20.286127 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:20.286233 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:20.294281 (+ 8048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.299366 (+ 5085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.302050 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.302712 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.303455 (+ 743us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.309941 (+ 6486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:20.309948 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:20.309952 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:20.310002 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:20.313582 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:20.314668 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:20.314674 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:20.314678 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:20.314853 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:20.315436 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:20.315441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:20.315444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:20.315809 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:20.315948 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:20.316952 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0929 15:54:20.317174 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35462,"thread_start_us":35337,"threads_started":1}
I20260929 15:54:20.319077 29530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.244416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:43589 (local address 127.15.196.195:35447)
0929 15:54:20.280573 (+ 36157us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:20.280642 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:20.283353 (+ 2711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:20.286779 (+ 3426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:20.286793 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:20.289413 (+ 2620us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:20.293991 (+ 4578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:20.294049 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.299582 (+ 5533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.299589 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:20.301821 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:20.301843 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.303815 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.303827 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:20.304195 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.306517 (+ 2322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:20.306555 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:20.309641 (+ 3086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:20.313753 (+ 4112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:20.313765 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:20.313786 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:20.314506 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:20.315009 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:20.315014 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:20.315018 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:20.315280 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:20.315983 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:20.315993 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:20.316478 (+ 485us) client_negotiation.cc:769] Sending connection context
0929 15:54:20.316807 (+ 329us) client_negotiation.cc:240] Negotiation successful
0929 15:54:20.317416 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35734,"thread_start_us":36707,"threads_started":1}
I20260929 15:54:20.320991 29527 heartbeater.cc:347] Connected to a master server at 127.15.196.254:43589
I20260929 15:54:20.321354 29527 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:20.322315 29527 heartbeater.cc:511] Master 127.15.196.254:43589 requested a full tablet report, sending...
I20260929 15:54:20.324266 29049 ts_manager.cc:194] Registered new tserver with Master: 044a5493c6724406a4f09ed58fa4d758 (127.15.196.195:32787)
I20260929 15:54:20.325623 29049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:35447
I20260929 15:54:20.336894 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:54:20.368459 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.351364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54352 (local address 127.15.196.254:43589)
0929 15:54:20.351573 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:20.351580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:20.352195 (+ 615us) server_negotiation.cc:407] Connection header received
0929 15:54:20.352409 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:20.352415 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:20.352510 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:20.352619 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:20.354750 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.356092 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.358090 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.358742 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.358891 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.363016 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:20.363021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:20.363024 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:20.363061 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:20.365741 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:20.366590 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:20.366594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:20.366597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:20.366723 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:20.367154 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:20.367158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:20.367160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:20.367475 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:20.367682 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:20.368054 (+ 372us) server_negotiation.cc:299] Negotiation successful
0929 15:54:20.368224 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 15:54:20.374622 29049 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:54:20.387488 29049 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:54:20.444620 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.404134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:35709 (local address 127.0.0.1:48326)
0929 15:54:20.430953 (+ 26819us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:20.431025 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:20.431322 (+ 297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:20.434592 (+ 3270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:20.434601 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:20.434645 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:20.435170 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:20.435185 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.438285 (+ 3100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.438291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:20.441170 (+ 2879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:20.441193 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.443577 (+ 2384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.443583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:20.443781 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.443861 (+ 80us) client_negotiation.cc:769] Sending connection context
0929 15:54:20.444006 (+ 145us) client_negotiation.cc:240] Negotiation successful
0929 15:54:20.444227 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26445,"thread_start_us":31404,"threads_started":1}
I20260929 15:54:20.444927 29554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.405967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38076 (local address 127.15.196.193:38219)
0929 15:54:20.419882 (+ 13915us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:20.419897 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:20.434800 (+ 14903us) server_negotiation.cc:407] Connection header received
0929 15:54:20.435179 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:20.435190 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:20.435515 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:20.435695 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:20.437607 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.439236 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.441993 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.443433 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.443601 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.443785 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:20.444376 (+ 591us) server_negotiation.cc:299] Negotiation successful
0929 15:54:20.444569 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13692,"thread_start_us":13498,"threads_started":1}
I20260929 15:54:20.445685 29552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.405438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48326 (local address 127.15.196.194:35709)
0929 15:54:20.433031 (+ 27593us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:20.433044 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:20.433084 (+ 40us) server_negotiation.cc:407] Connection header received
0929 15:54:20.433215 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:20.433227 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:20.433587 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:20.433835 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:20.436174 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.438099 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.441405 (+ 3306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.443006 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.443208 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.443424 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:20.445165 (+ 1741us) server_negotiation.cc:299] Negotiation successful
0929 15:54:20.445341 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27372,"thread_start_us":27175,"threads_started":1}
I20260929 15:54:20.445989 29550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.404624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:38219 (local address 127.0.0.1:38076)
0929 15:54:20.434630 (+ 30006us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:20.434661 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:20.434941 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:20.436103 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:20.436107 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:20.436129 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:20.436488 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:20.436501 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.439563 (+ 3062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.439567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:20.441812 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:20.441831 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.443996 (+ 2165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.444002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:20.444158 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.444190 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 15:54:20.445397 (+ 1207us) client_negotiation.cc:240] Negotiation successful
0929 15:54:20.445552 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29821,"thread_start_us":29641,"threads_started":1}
I20260929 15:54:20.448521 29553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.405581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43844 (local address 127.15.196.195:32787)
0929 15:54:20.430110 (+ 24529us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:20.430125 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:20.435978 (+ 5853us) server_negotiation.cc:407] Connection header received
0929 15:54:20.436758 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:20.436765 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:20.437035 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:20.437227 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:20.437869 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.440916 (+ 3047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.444541 (+ 3625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.446361 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.446541 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.446701 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:20.448016 (+ 1315us) server_negotiation.cc:299] Negotiation successful
0929 15:54:20.448187 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23922,"thread_start_us":24075,"threads_started":1}
I20260929 15:54:20.449256 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.404725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:32787 (local address 127.0.0.1:43844)
0929 15:54:20.435818 (+ 31093us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:20.435837 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:20.435976 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:20.437217 (+ 1241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:20.437222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:20.437246 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:20.437634 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:20.437649 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.441087 (+ 3438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.441093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:20.444374 (+ 3281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:20.444390 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.447186 (+ 2796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.447198 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:20.447409 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.447754 (+ 345us) client_negotiation.cc:769] Sending connection context
0929 15:54:20.447837 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 15:54:20.448909 (+ 1072us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30958,"mutex_wait_us":1,"thread_start_us":26879,"threads_started":1}
I20260929 15:54:20.450191 29327 tablet_service.cc:1467] Processing CreateTablet for tablet 13a86c2c266b4067aa6f77026917b96e (DEFAULT_TABLE table=test-table [id=44f0a02cdd45438e8bd9660e85585840]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:54:20.450191 29182 tablet_service.cc:1467] Processing CreateTablet for tablet 13a86c2c266b4067aa6f77026917b96e (DEFAULT_TABLE table=test-table [id=44f0a02cdd45438e8bd9660e85585840]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:54:20.451967 29327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13a86c2c266b4067aa6f77026917b96e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:20.453662 29182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13a86c2c266b4067aa6f77026917b96e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:20.453343 29462 tablet_service.cc:1467] Processing CreateTablet for tablet 13a86c2c266b4067aa6f77026917b96e (DEFAULT_TABLE table=test-table [id=44f0a02cdd45438e8bd9660e85585840]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:54:20.455201 29462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13a86c2c266b4067aa6f77026917b96e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:20.487257 29557 tablet_bootstrap.cc:492] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758: Bootstrap starting.
I20260929 15:54:20.494940 29555 tablet_bootstrap.cc:492] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed: Bootstrap starting.
I20260929 15:54:20.497156 29557 tablet_bootstrap.cc:654] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:20.498661 29557 log.cc:824] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:20.499094 29555 tablet_bootstrap.cc:654] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:20.500097 29556 tablet_bootstrap.cc:492] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568: Bootstrap starting.
I20260929 15:54:20.500553 29555 log.cc:824] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:20.504640 29556 tablet_bootstrap.cc:654] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:20.506078 29557 tablet_bootstrap.cc:492] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758: No bootstrap required, opened a new log
I20260929 15:54:20.506088 29556 log.cc:824] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:20.506479 29557 ts_tablet_manager.cc:1397] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.010s
I20260929 15:54:20.506714 29555 tablet_bootstrap.cc:492] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed: No bootstrap required, opened a new log
I20260929 15:54:20.507078 29555 ts_tablet_manager.cc:1397] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260929 15:54:20.518296 29557 raft_consensus.cc:348] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300f6abdc7541dbb273118123588aed" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 35709 } } peers { permanent_uuid: "bba41180083046dfbf4926c947fdd568" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38219 } } peers { permanent_uuid: "044a5493c6724406a4f09ed58fa4d758" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 32787 } }
I20260929 15:54:20.518970 29557 raft_consensus.cc:374] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:20.519153 29557 raft_consensus.cc:729] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 044a5493c6724406a4f09ed58fa4d758, State: Initialized, Role: FOLLOWER
I20260929 15:54:20.520380 29557 consensus_queue.cc:260] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300f6abdc7541dbb273118123588aed" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 35709 } } peers { permanent_uuid: "bba41180083046dfbf4926c947fdd568" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38219 } } peers { permanent_uuid: "044a5493c6724406a4f09ed58fa4d758" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 32787 } }
I20260929 15:54:20.524451 29527 heartbeater.cc:503] Master 127.15.196.254:43589 was elected leader, sending a full tablet report...
I20260929 15:54:20.523233 29555 raft_consensus.cc:348] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300f6abdc7541dbb273118123588aed" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 35709 } } peers { permanent_uuid: "bba41180083046dfbf4926c947fdd568" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38219 } } peers { permanent_uuid: "044a5493c6724406a4f09ed58fa4d758" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 32787 } }
I20260929 15:54:20.524906 29555 raft_consensus.cc:374] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:20.525020 29555 raft_consensus.cc:729] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b300f6abdc7541dbb273118123588aed, State: Initialized, Role: FOLLOWER
I20260929 15:54:20.525305 29557 ts_tablet_manager.cc:1428] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758: Time spent starting tablet: real 0.019s user 0.003s sys 0.012s
I20260929 15:54:20.525797 29555 consensus_queue.cc:260] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300f6abdc7541dbb273118123588aed" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 35709 } } peers { permanent_uuid: "bba41180083046dfbf4926c947fdd568" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38219 } } peers { permanent_uuid: "044a5493c6724406a4f09ed58fa4d758" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 32787 } }
I20260929 15:54:20.526640 29248 heartbeater.cc:503] Master 127.15.196.254:43589 was elected leader, sending a full tablet report...
I20260929 15:54:20.527591 29556 tablet_bootstrap.cc:492] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568: No bootstrap required, opened a new log
I20260929 15:54:20.527868 29556 ts_tablet_manager.cc:1397] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.024s
I20260929 15:54:20.533949 29555 ts_tablet_manager.cc:1428] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed: Time spent starting tablet: real 0.027s user 0.021s sys 0.000s
I20260929 15:54:20.539408 29556 raft_consensus.cc:348] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300f6abdc7541dbb273118123588aed" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 35709 } } peers { permanent_uuid: "bba41180083046dfbf4926c947fdd568" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38219 } } peers { permanent_uuid: "044a5493c6724406a4f09ed58fa4d758" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 32787 } }
I20260929 15:54:20.539871 29556 raft_consensus.cc:374] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:20.539973 29556 raft_consensus.cc:729] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bba41180083046dfbf4926c947fdd568, State: Initialized, Role: FOLLOWER
I20260929 15:54:20.540613 29556 consensus_queue.cc:260] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300f6abdc7541dbb273118123588aed" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 35709 } } peers { permanent_uuid: "bba41180083046dfbf4926c947fdd568" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38219 } } peers { permanent_uuid: "044a5493c6724406a4f09ed58fa4d758" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 32787 } }
I20260929 15:54:20.542364 29556 ts_tablet_manager.cc:1428] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260929 15:54:20.601310 29563 raft_consensus.cc:482] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:54:20.601682 29563 raft_consensus.cc:504] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300f6abdc7541dbb273118123588aed" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 35709 } } peers { permanent_uuid: "bba41180083046dfbf4926c947fdd568" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38219 } } peers { permanent_uuid: "044a5493c6724406a4f09ed58fa4d758" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 32787 } }
I20260929 15:54:20.603689 29563 leader_election.cc:288] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b300f6abdc7541dbb273118123588aed (127.15.196.194:35709), 044a5493c6724406a4f09ed58fa4d758 (127.15.196.195:32787)
I20260929 15:54:20.612465 29552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.604546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:34663 (local address 127.15.196.194:35709)
0929 15:54:20.604730 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:20.604735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:20.605524 (+ 789us) server_negotiation.cc:407] Connection header received
0929 15:54:20.605785 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:20.605793 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:20.605898 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:20.605993 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:20.606729 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.607885 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.610483 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.611400 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.611594 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.611685 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:20.612074 (+ 389us) server_negotiation.cc:299] Negotiation successful
0929 15:54:20.612219 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 15:54:20.612661 29553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.604202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:35201 (local address 127.15.196.195:32787)
0929 15:54:20.604480 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:20.604490 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:20.605369 (+ 879us) server_negotiation.cc:407] Connection header received
0929 15:54:20.605601 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:20.605605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:20.605717 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:20.605823 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:20.606447 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.607391 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.610244 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.611452 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.611625 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.611695 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:20.612246 (+ 551us) server_negotiation.cc:299] Negotiation successful
0929 15:54:20.612412 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 15:54:20.613103 29564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.604113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:35709 (local address 127.15.196.193:34663)
0929 15:54:20.605419 (+ 1306us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:20.605440 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:20.605647 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:20.606029 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:20.606034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:20.606062 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:20.606508 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:20.606525 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.608023 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.608030 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:20.610327 (+ 2297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:20.610337 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.611633 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.611640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:20.611867 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.611909 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 15:54:20.612018 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 15:54:20.612173 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1195,"mutex_wait_us":45,"thread_start_us":613,"threads_started":1}
I20260929 15:54:20.613276 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.603977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:32787 (local address 127.15.196.193:35201)
0929 15:54:20.605189 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:20.605210 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:20.605409 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:20.605841 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:20.605846 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:20.605869 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:20.606270 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:20.606284 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.607530 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.607536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:20.610063 (+ 2527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:20.610078 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.611861 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.611867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:20.612060 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.612092 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 15:54:20.612744 (+ 652us) client_negotiation.cc:240] Negotiation successful
0929 15:54:20.612877 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"mutex_wait_us":93,"thread_start_us":1125,"threads_started":1}
I20260929 15:54:20.614241 29482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13a86c2c266b4067aa6f77026917b96e" candidate_uuid: "bba41180083046dfbf4926c947fdd568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "044a5493c6724406a4f09ed58fa4d758" is_pre_election: true
I20260929 15:54:20.614245 29347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13a86c2c266b4067aa6f77026917b96e" candidate_uuid: "bba41180083046dfbf4926c947fdd568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b300f6abdc7541dbb273118123588aed" is_pre_election: true
I20260929 15:54:20.614806 29482 raft_consensus.cc:2454] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bba41180083046dfbf4926c947fdd568 in term 0.
I20260929 15:54:20.614809 29347 raft_consensus.cc:2454] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bba41180083046dfbf4926c947fdd568 in term 0.
I20260929 15:54:20.615525 29137 leader_election.cc:302] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 044a5493c6724406a4f09ed58fa4d758, bba41180083046dfbf4926c947fdd568; no voters:
I20260929 15:54:20.616009 29563 raft_consensus.cc:2781] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:54:20.616189 29563 raft_consensus.cc:482] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:54:20.616328 29563 raft_consensus.cc:3037] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:20.618680 29563 raft_consensus.cc:504] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300f6abdc7541dbb273118123588aed" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 35709 } } peers { permanent_uuid: "bba41180083046dfbf4926c947fdd568" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38219 } } peers { permanent_uuid: "044a5493c6724406a4f09ed58fa4d758" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 32787 } }
I20260929 15:54:20.619426 29563 leader_election.cc:288] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [CANDIDATE]: Term 1 election: Requested vote from peers b300f6abdc7541dbb273118123588aed (127.15.196.194:35709), 044a5493c6724406a4f09ed58fa4d758 (127.15.196.195:32787)
I20260929 15:54:20.619902 29347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13a86c2c266b4067aa6f77026917b96e" candidate_uuid: "bba41180083046dfbf4926c947fdd568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b300f6abdc7541dbb273118123588aed"
I20260929 15:54:20.619925 29482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13a86c2c266b4067aa6f77026917b96e" candidate_uuid: "bba41180083046dfbf4926c947fdd568" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "044a5493c6724406a4f09ed58fa4d758"
I20260929 15:54:20.620116 29347 raft_consensus.cc:3037] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:20.620122 29482 raft_consensus.cc:3037] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:20.622431 29347 raft_consensus.cc:2454] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bba41180083046dfbf4926c947fdd568 in term 1.
I20260929 15:54:20.622864 29136 leader_election.cc:302] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b300f6abdc7541dbb273118123588aed, bba41180083046dfbf4926c947fdd568; no voters:
I20260929 15:54:20.623100 29482 raft_consensus.cc:2454] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bba41180083046dfbf4926c947fdd568 in term 1.
I20260929 15:54:20.623162 29563 raft_consensus.cc:2781] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:54:20.624475 29563 raft_consensus.cc:686] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [term 1 LEADER]: Becoming Leader. State: Replica: bba41180083046dfbf4926c947fdd568, State: Running, Role: LEADER
I20260929 15:54:20.625059 29563 consensus_queue.cc:237] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300f6abdc7541dbb273118123588aed" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 35709 } } peers { permanent_uuid: "bba41180083046dfbf4926c947fdd568" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38219 } } peers { permanent_uuid: "044a5493c6724406a4f09ed58fa4d758" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 32787 } }
I20260929 15:54:20.629910 29050 catalog_manager.cc:5495] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 reported cstate change: term changed from 0 to 1, leader changed from <none> to bba41180083046dfbf4926c947fdd568 (127.15.196.193). New cstate: current_term: 1 leader_uuid: "bba41180083046dfbf4926c947fdd568" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b300f6abdc7541dbb273118123588aed" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 35709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bba41180083046dfbf4926c947fdd568" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "044a5493c6724406a4f09ed58fa4d758" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 32787 } health_report { overall_health: UNKNOWN } } }
I20260929 15:54:20.684260 29554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.678420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38078 (local address 127.15.196.193:38219)
0929 15:54:20.678631 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:20.678636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:20.678656 (+ 20us) server_negotiation.cc:407] Connection header received
0929 15:54:20.678923 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:20.678927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:20.679008 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:20.679164 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:54:20.679564 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.680337 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.681827 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.682228 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.682372 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.682964 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:54:20.683652 (+ 688us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:54:20.683820 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:20.683909 (+ 89us) server_negotiation.cc:299] Negotiation successful
0929 15:54:20.683989 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 15:54:20.693437 29347 raft_consensus.cc:1261] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed [term 1 FOLLOWER]: Refusing update from remote peer bba41180083046dfbf4926c947fdd568: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:54:20.693518 29482 raft_consensus.cc:1261] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758 [term 1 FOLLOWER]: Refusing update from remote peer bba41180083046dfbf4926c947fdd568: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:54:20.694587 29566 consensus_queue.cc:1035] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b300f6abdc7541dbb273118123588aed" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 35709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:54:20.695050 29563 consensus_queue.cc:1035] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "044a5493c6724406a4f09ed58fa4d758" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 32787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 15:54:20.696177 29393 tablet.cc:2369] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:54:20.707201 29569 mvcc.cc:205] Tried to move back new op lower bound from 7334695979789193216 to 7334695979524694016. Current Snapshot: MvccSnapshot[applied={T|T < 7334695979789193216}]
I20260929 15:54:20.728355 29571 mvcc.cc:205] Tried to move back new op lower bound from 7334695979789193216 to 7334695979524694016. Current Snapshot: MvccSnapshot[applied={T|T < 7334695979789193216}]
I20260929 15:54:20.742350 29049 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:54:20.742688 29049 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:54:20.747658 29049 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 90e1706e1e8e46edae8948988270c04f: Sending DeleteTablet for 3 replicas of tablet 13a86c2c266b4067aa6f77026917b96e
I20260929 15:54:20.748939 29327 tablet_service.cc:1514] Processing DeleteTablet for tablet 13a86c2c266b4067aa6f77026917b96e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:54:20 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:48326
I20260929 15:54:20.748926 29182 tablet_service.cc:1514] Processing DeleteTablet for tablet 13a86c2c266b4067aa6f77026917b96e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:54:20 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:38076
I20260929 15:54:20.750224 29462 tablet_service.cc:1514] Processing DeleteTablet for tablet 13a86c2c266b4067aa6f77026917b96e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:54:20 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:43844
I20260929 15:54:20.751639 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 29087
I20260929 15:54:20.767333 29580 tablet_replica.cc:330] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed: stopping tablet replica
I20260929 15:54:20.767848 29580 raft_consensus.cc:2229] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:54:20.768644 29580 raft_consensus.cc:2258] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:54:20.771906 29579 tablet_replica.cc:330] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568: stopping tablet replica
I20260929 15:54:20.772377 29579 raft_consensus.cc:2229] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:54:20.773018 29579 raft_consensus.cc:2258] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:54:20.774245 29581 tablet_replica.cc:330] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758: stopping tablet replica
I20260929 15:54:20.774667 29581 raft_consensus.cc:2229] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:54:20.775213 29581 raft_consensus.cc:2258] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:54:20.777341 29581 ts_tablet_manager.cc:1905] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:54:20.780974 29554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.756762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38094 (local address 127.15.196.193:38219)
0929 15:54:20.756941 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:20.756950 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:20.756976 (+ 26us) server_negotiation.cc:407] Connection header received
0929 15:54:20.757045 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:20.757052 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:20.757149 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:20.757273 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:20.762679 (+ 5406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.764322 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.766755 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.767350 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.767492 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.771346 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:20.771364 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:20.771385 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:20.771445 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:20.776516 (+ 5071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:20.777459 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:20.777469 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:20.777472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:20.777627 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:20.778066 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:20.778092 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:20.778097 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:20.778545 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:20.778796 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:20.780344 (+ 1548us) server_negotiation.cc:299] Negotiation successful
0929 15:54:20.780600 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 15:54:20.781669 29243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:20.785082 29581 ts_tablet_manager.cc:1918] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:54:20.785257 29581 log.cc:1196] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/13a86c2c266b4067aa6f77026917b96e
I20260929 15:54:20.785768 29581 ts_tablet_manager.cc:1939] T 13a86c2c266b4067aa6f77026917b96e P 044a5493c6724406a4f09ed58fa4d758: Deleting consensus metadata
I20260929 15:54:20.786347 29580 ts_tablet_manager.cc:1905] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:54:20.787943 29036 catalog_manager.cc:4841] TS 044a5493c6724406a4f09ed58fa4d758 (127.15.196.195:32787): tablet 13a86c2c266b4067aa6f77026917b96e (table test-table [id=44f0a02cdd45438e8bd9660e85585840]) successfully deleted
I20260929 15:54:20.791244 29580 ts_tablet_manager.cc:1918] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:54:20.791524 29580 log.cc:1196] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/13a86c2c266b4067aa6f77026917b96e
I20260929 15:54:20.792043 29580 ts_tablet_manager.cc:1939] T 13a86c2c266b4067aa6f77026917b96e P b300f6abdc7541dbb273118123588aed: Deleting consensus metadata
I20260929 15:54:20.794019 29035 catalog_manager.cc:4841] TS b300f6abdc7541dbb273118123588aed (127.15.196.194:35709): tablet 13a86c2c266b4067aa6f77026917b96e (table test-table [id=44f0a02cdd45438e8bd9660e85585840]) successfully deleted
I20260929 15:54:20.882726 29579 ts_tablet_manager.cc:1905] T 13a86c2c266b4067aa6f77026917b96e P bba41180083046dfbf4926c947fdd568: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:54:20.884646 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 29087
I20260929 15:54:20.903086 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 29252
W20260929 15:54:20.903206 29037 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.196.193:38219 (error 108)
I20260929 15:54:20.905023 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.904650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:38219 (local address 127.0.0.1:38100)
0929 15:54:20.904803 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:20.904895 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260929 15:54:20.905237 29037 catalog_manager.cc:4584] TS bba41180083046dfbf4926c947fdd568 (127.15.196.193:38219): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 13a86c2c266b4067aa6f77026917b96e: Network error: Client connection negotiation failed: client connection to 127.15.196.193:38219: connect: Connection refused (error 111)
I20260929 15:54:20.917111 29552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.904026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48328 (local address 127.15.196.194:35709)
0929 15:54:20.904245 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:20.904250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:20.904356 (+ 106us) server_negotiation.cc:407] Connection header received
0929 15:54:20.904646 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:20.904666 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:20.904753 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:20.904886 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:20.906296 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.907092 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.908479 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:20.908871 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:20.909010 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:20.911609 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:20.911621 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:20.911636 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:20.911675 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:20.914334 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:20.915061 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:20.915070 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:20.915072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:20.915207 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:20.915591 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:20.915598 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:20.915601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:20.915938 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:20.916270 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:20.916554 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 15:54:20.916767 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260929 15:54:20.917800 29387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:20.958740 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.958256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:38219 (local address 127.0.0.1:38116)
0929 15:54:20.958467 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:20.958597 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260929 15:54:20.959874 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:20.959514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:38219 (local address 127.0.0.1:38118)
0929 15:54:20.959679 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:20.959758 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260929 15:54:21.012600 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 29252
I20260929 15:54:21.020444 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:21.019964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:38219 (local address 127.0.0.1:38132)
0929 15:54:21.020177 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:21.020275 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260929 15:54:21.021476 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:21.021150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:38219 (local address 127.0.0.1:38140)
0929 15:54:21.021296 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:21.021369 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260929 15:54:21.030654 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 29396
I20260929 15:54:21.044337 29553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:21.031623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43854 (local address 127.15.196.195:32787)
0929 15:54:21.031811 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:21.031816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:21.031834 (+ 18us) server_negotiation.cc:407] Connection header received
0929 15:54:21.031973 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:21.031977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:21.032051 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:21.032134 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:21.033376 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:21.034093 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:21.035431 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:21.035833 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:21.035937 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:21.038438 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:21.038462 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:21.038493 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:21.038567 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:21.041756 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:21.042375 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:21.042383 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:21.042385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:21.042536 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:21.042851 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:21.042854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:21.042856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:21.043183 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:21.043532 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:21.043796 (+ 264us) server_negotiation.cc:299] Negotiation successful
0929 15:54:21.043953 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 15:54:21.044975 29522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:21.102058 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:21.101571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:38219 (local address 127.0.0.1:38142)
0929 15:54:21.101791 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:21.101881 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260929 15:54:21.103076 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:21.102701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:38219 (local address 127.0.0.1:38146)
0929 15:54:21.102902 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:21.102969 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260929 15:54:21.134083 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 29396
I20260929 15:54:21.150563 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 29017
I20260929 15:54:21.151960 29080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:21.266299 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 29017
I20260929 15:54:21.291523 16147 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 15:54:21.626973 16147 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 15:54:21.627233 16147 mini_ranger.cc:66] Stopping Ranger...
I20260929 15:54:21.972191 16147 mini_ranger.cc:68] Stopped Ranger
2026-09-29 15:54:21.972 UTC [28069] LOG: received smart shutdown request
2026-09-29 15:54:21.975 UTC [28069] LOG: background worker "logical replication launcher" (PID 28078) exited with exit code 1
2026-09-29 15:54:21.976 UTC [28073] LOG: shutting down
2026-09-29 15:54:22.126 UTC [28069] LOG: database system is shut down
2026-09-29T15:54:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52913 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:54:22 dist-test-slave-c51v krb5kdc[29595](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:54:22 dist-test-slave-c51v krb5kdc[29595](info): set up 2 sockets
Sep 29 15:54:22 dist-test-slave-c51v krb5kdc[29595](info): commencing operation
krb5kdc: starting...
W20260929 15:54:23.361800 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:54:23 dist-test-slave-c51v krb5kdc[29595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:54:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:54:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:54:23.594837 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:38137
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:35137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:38137,127.15.196.253:33725,127.15.196.252:40773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:54:23.922796 29608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:23.923141 29608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:23.923337 29608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:23.930694 29608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:54:23.930852 29608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:23.930960 29608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:23.931075 29608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:54:23.931169 29608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:54:23.941442 29608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:35137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:38137,127.15.196.253:33725,127.15.196.252:40773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:38137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:23.942862 29608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:23.944290 29608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:23.956141 29616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:23.956239 29613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:23.956236 29614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:23.957229 29608 server_base.cc:1034] running on GCE node
I20260929 15:54:23.958060 29608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:23.959581 29608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:23.960814 29608 hybrid_clock.cc:648] HybridClock initialized: now 1790697263960745 us; error 54 us; skew 500 ppm
I20260929 15:54:23.961177 29608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:23 dist-test-slave-c51v krb5kdc[29595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697263, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:23.964555 29608 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:54:23.966140 29608 webserver.cc:460] Webserver started at http://127.15.196.254:44381/ using document root <none> and password file <none>
I20260929 15:54:23.966828 29608 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:23.966929 29608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:23.967190 29608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:54:23.970666 29608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "6d9ab46751154c709e7d9d48be4dd66e"
format_stamp: "Formatted at 2026-09-29 15:54:23 on dist-test-slave-c51v"
server_key: "16510c476257a233dadab1fe5ed5abe9"
server_key_iv: "01599447a770a868038c19b2b721687f"
server_key_version: "encryptionkey@0"
I20260929 15:54:23.971743 29608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "6d9ab46751154c709e7d9d48be4dd66e"
format_stamp: "Formatted at 2026-09-29 15:54:23 on dist-test-slave-c51v"
server_key: "16510c476257a233dadab1fe5ed5abe9"
server_key_iv: "01599447a770a868038c19b2b721687f"
server_key_version: "encryptionkey@0"
I20260929 15:54:23.977069 29608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 15:54:23.980440 29623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:23.981817 29608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 15:54:23.981956 29608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "6d9ab46751154c709e7d9d48be4dd66e"
format_stamp: "Formatted at 2026-09-29 15:54:23 on dist-test-slave-c51v"
server_key: "16510c476257a233dadab1fe5ed5abe9"
server_key_iv: "01599447a770a868038c19b2b721687f"
server_key_version: "encryptionkey@0"
I20260929 15:54:23.982120 29608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:23.997217 29608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:24.001365 29608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:24.001605 29608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:24.020449 29608 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:38137
I20260929 15:54:24.020483 29674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:38137 every 8 connection(s)
I20260929 15:54:24.021891 29608 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:54:24.025707 29675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:24.026798 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 29608
I20260929 15:54:24.027024 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:54:24.027760 16147 external_mini_cluster.cc:1428] Setting key 3c7b266d487d8819f0f09bd474ff81c3
I20260929 15:54:24.032595 29675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.196.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.253@KRBTEST.COM" created.
Sep 29 15:54:24 dist-test-slave-c51v krb5kdc[29595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697263, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:24.063112 29679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.035985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35188 (local address 127.15.196.254:38137)
0929 15:54:24.038127 (+ 2142us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:24.038133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:24.038177 (+ 44us) server_negotiation.cc:407] Connection header received
0929 15:54:24.040687 (+ 2510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:24.040693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:24.040942 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:24.041109 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:24.042800 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:24.044084 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:24.046454 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:24.047015 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:24.047154 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:24.053841 (+ 6687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:24.053857 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:24.053880 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:24.053957 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:24.058426 (+ 4469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:24.059161 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:24.059173 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:24.059180 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:24.059478 (+ 298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:24.059875 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:24.059883 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:24.059890 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:24.060691 (+ 801us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:24.060932 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:24.061583 (+ 651us) server_negotiation.cc:299] Negotiation successful
0929 15:54:24.061958 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":1903,"spinlock_wait_cycles":17152,"thread_start_us":1657,"threads_started":1}
I20260929 15:54:24.063151 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.035985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38137 (local address 127.0.0.1:35188)
0929 15:54:24.036987 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.037047 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:24.040258 (+ 3211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:24.041196 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:24.041206 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:24.042044 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:24.042581 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:24.042607 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:24.044246 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:24.044255 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:24.046290 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:24.046310 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:24.047276 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:24.047306 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:24.047561 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:24.048318 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:24.048351 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:24.053549 (+ 5198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:24.058586 (+ 5037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:24.058597 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:24.058607 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:24.059015 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:24.059605 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:24.059608 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:24.059610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:24.059771 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:24.060869 (+ 1098us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:24.060880 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:24.061159 (+ 279us) client_negotiation.cc:769] Sending connection context
0929 15:54:24.061492 (+ 333us) client_negotiation.cc:240] Negotiation successful
0929 15:54:24.062137 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":103,"thread_start_us":426,"threads_started":1}
I20260929 15:54:24.066355 29675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } has no permanent_uuid. Determining permanent_uuid...
I20260929 15:54:24.067721 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.067226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:33725 (local address 127.0.0.1:49704)
0929 15:54:24.067497 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.067608 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:33725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260929 15:54:24.068017 29625 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.196.253:33725: connect: Connection refused (error 111)
I20260929 15:54:24.069607 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.069247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:33725 (local address 127.0.0.1:49710)
0929 15:54:24.069416 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.069481 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:33725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260929 15:54:24.069828 29675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.253:33725: Network error: Client connection negotiation failed: client connection to 127.15.196.253:33725: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.253.keytab.
Entry for principal kudu/127.15.196.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.253.keytab.
I20260929 15:54:24.083094 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.253:33725
--webserver_interface=127.15.196.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.253.keytab
--principal=kudu/127.15.196.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:35137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:38137,127.15.196.253:33725,127.15.196.252:40773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:54:24.119160 29675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } attempt: 1
I20260929 15:54:24.120409 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.119914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:33725 (local address 127.0.0.1:49724)
0929 15:54:24.120119 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.120260 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:33725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260929 15:54:24.121544 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.121165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:33725 (local address 127.0.0.1:49734)
0929 15:54:24.121340 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.121410 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:33725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260929 15:54:24.121740 29675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.253:33725: Network error: Client connection negotiation failed: client connection to 127.15.196.253:33725: connect: Connection refused (error 111)
I20260929 15:54:24.189966 29675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } attempt: 2
I20260929 15:54:24.191231 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.190733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:33725 (local address 127.0.0.1:49748)
0929 15:54:24.190986 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.191092 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:33725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260929 15:54:24.192292 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.191928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:33725 (local address 127.0.0.1:49752)
0929 15:54:24.192099 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.192162 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:33725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 15:54:24.192471 29675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.253:33725: Network error: Client connection negotiation failed: client connection to 127.15.196.253:33725: connect: Connection refused (error 111)
I20260929 15:54:24.283707 29675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } attempt: 3
I20260929 15:54:24.284921 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.284367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:33725 (local address 127.0.0.1:49762)
0929 15:54:24.284609 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.284747 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:33725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260929 15:54:24.286125 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.285629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:33725 (local address 127.0.0.1:49766)
0929 15:54:24.285837 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.285945 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:33725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260929 15:54:24.286332 29675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.253:33725: Network error: Client connection negotiation failed: client connection to 127.15.196.253:33725: connect: Connection refused (error 111)
W20260929 15:54:24.409756 29682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:24.410063 29682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:24.410154 29682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:24.417457 29682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:54:24.417573 29682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:24.417620 29682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:24.417706 29682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:54:24.417775 29682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:54:24.427670 29682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:35137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:38137,127.15.196.253:33725,127.15.196.252:40773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.253.keytab
--principal=kudu/127.15.196.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.253:33725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.196.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:24.429095 29682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:24.429579 29675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } attempt: 4
I20260929 15:54:24.430800 29682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 15:54:24.430837 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.430227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:33725 (local address 127.0.0.1:49776)
0929 15:54:24.430565 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.430675 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:33725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
I20260929 15:54:24.431924 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.431511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:33725 (local address 127.0.0.1:49790)
0929 15:54:24.431667 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.431768 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:33725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260929 15:54:24.432121 29675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.253:33725: Network error: Client connection negotiation failed: client connection to 127.15.196.253:33725: connect: Connection refused (error 111)
W20260929 15:54:24.443339 29687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:24.443276 29690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:24.443384 29688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:24.448846 29682 server_base.cc:1034] running on GCE node
I20260929 15:54:24.449638 29682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:24.451133 29682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:24.452368 29682 hybrid_clock.cc:648] HybridClock initialized: now 1790697264452316 us; error 61 us; skew 500 ppm
I20260929 15:54:24.452725 29682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:24 dist-test-slave-c51v krb5kdc[29595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:24.455741 29682 init.cc:377] Logged in from keytab as kudu/127.15.196.253@KRBTEST.COM (short username kudu)
I20260929 15:54:24.457271 29682 webserver.cc:460] Webserver started at http://127.15.196.253:41245/ using document root <none> and password file <none>
I20260929 15:54:24.458001 29682 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:24.458158 29682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:24.458467 29682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:54:24.461506 29682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/data/instance:
uuid: "0deb7499d933420f99eebcf8aef865db"
format_stamp: "Formatted at 2026-09-29 15:54:24 on dist-test-slave-c51v"
server_key: "0f6a5444a8a28c1515e806235118ef2f"
server_key_iv: "f82d651c2455c069f26f8b2bab3b5c13"
server_key_version: "encryptionkey@0"
I20260929 15:54:24.462317 29682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/wal/instance:
uuid: "0deb7499d933420f99eebcf8aef865db"
format_stamp: "Formatted at 2026-09-29 15:54:24 on dist-test-slave-c51v"
server_key: "0f6a5444a8a28c1515e806235118ef2f"
server_key_iv: "f82d651c2455c069f26f8b2bab3b5c13"
server_key_version: "encryptionkey@0"
I20260929 15:54:24.467608 29682 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 15:54:24.470855 29697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:24.472178 29682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:54:24.472364 29682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/wal
uuid: "0deb7499d933420f99eebcf8aef865db"
format_stamp: "Formatted at 2026-09-29 15:54:24 on dist-test-slave-c51v"
server_key: "0f6a5444a8a28c1515e806235118ef2f"
server_key_iv: "f82d651c2455c069f26f8b2bab3b5c13"
server_key_version: "encryptionkey@0"
I20260929 15:54:24.472579 29682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:24.496685 29682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:24.500816 29682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:24.501173 29682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:24.519905 29682 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.253:33725
I20260929 15:54:24.519925 29748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.253:33725 every 8 connection(s)
I20260929 15:54:24.521345 29682 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/data/info.pb
I20260929 15:54:24.525245 29749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:24.525408 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 29682
I20260929 15:54:24.525605 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-1/wal/instance
I20260929 15:54:24.526157 16147 external_mini_cluster.cc:1428] Setting key 25407e6e8288a63f3fc22c097b32c505
I20260929 15:54:24.535430 29749 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.196.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.252@KRBTEST.COM" created.
Sep 29 15:54:24 dist-test-slave-c51v krb5kdc[29595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.253@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:24.558938 29679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.538237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35202 (local address 127.15.196.254:38137)
0929 15:54:24.538419 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:24.538423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:24.539886 (+ 1463us) server_negotiation.cc:407] Connection header received
0929 15:54:24.541798 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:24.541804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:24.541899 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:24.542015 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:24.543674 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:24.544465 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:24.546032 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:24.546469 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:24.546627 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:24.551840 (+ 5213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:24.551851 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:24.551859 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:24.551911 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:24.555700 (+ 3789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:24.556495 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:24.556504 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:24.556511 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:24.556679 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:24.557090 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:24.557097 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:24.557104 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:24.557503 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:24.557684 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:24.558528 (+ 844us) server_negotiation.cc:299] Negotiation successful
0929 15:54:24.558714 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 15:54:24.560626 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.538867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38137 (local address 127.0.0.1:35202)
0929 15:54:24.539717 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.539755 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:24.541519 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:24.542276 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:24.542285 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:24.542808 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:24.543485 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:24.543505 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:24.544642 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:24.544646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:24.545877 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:24.545899 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:24.546717 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:24.546724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:24.546954 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:24.547756 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:24.547780 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:24.551584 (+ 3804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:24.555862 (+ 4278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:24.555870 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:24.555885 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:24.556347 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:24.556789 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:24.556792 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:24.556794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:24.556970 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:24.557635 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:24.557641 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:24.558057 (+ 416us) client_negotiation.cc:769] Sending connection context
0929 15:54:24.558430 (+ 373us) client_negotiation.cc:240] Negotiation successful
0929 15:54:24.559058 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":391,"threads_started":1}
I20260929 15:54:24.562433 29749 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:54:24 dist-test-slave-c51v krb5kdc[29595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.253@KRBTEST.COM for kudu/127.15.196.253@KRBTEST.COM
I20260929 15:54:24.578142 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.563252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:33725 (local address 127.0.0.1:49798)
0929 15:54:24.563435 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.563450 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:24.563574 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:24.564808 (+ 1234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:24.564812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:24.565261 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:24.565391 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:24.565398 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:24.566696 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:24.566703 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:24.567632 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:24.567644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:24.568490 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:24.568498 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:24.568777 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:24.569388 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:24.569418 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:24.572065 (+ 2647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:24.575478 (+ 3413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:24.575483 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:24.575487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:24.575869 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:24.576291 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:24.576298 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:24.576302 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:24.576476 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:24.577130 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:24.577135 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:24.577283 (+ 148us) client_negotiation.cc:769] Sending connection context
0929 15:54:24.577428 (+ 145us) client_negotiation.cc:240] Negotiation successful
0929 15:54:24.577721 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260929 15:54:24.578227 29754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.563575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.15.196.253:33725)
0929 15:54:24.564365 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:24.564372 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:24.564397 (+ 25us) server_negotiation.cc:407] Connection header received
0929 15:54:24.564437 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:24.564442 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:24.564659 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:24.564827 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:24.565540 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:24.566564 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:24.567764 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:24.568300 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:24.568452 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:24.572311 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:24.572328 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:24.572349 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:24.572396 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:24.575246 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:24.576002 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:24.576009 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:24.576011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:24.576169 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:24.576604 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:24.576609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:24.576614 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:24.576995 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:24.577211 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:24.577632 (+ 421us) server_negotiation.cc:299] Negotiation successful
0929 15:54:24.577849 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":414,"threads_started":1}
I20260929 15:54:24.580945 29749 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.252.keytab.
Entry for principal kudu/127.15.196.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.252.keytab.
I20260929 15:54:24.582264 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.581693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:51866)
0929 15:54:24.581892 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.582051 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260929 15:54:24.582536 29700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
I20260929 15:54:24.582940 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.252:40773
--webserver_interface=127.15.196.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.252.keytab
--principal=kudu/127.15.196.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:35137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:38137,127.15.196.253:33725,127.15.196.252:40773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:54:24.584232 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.583767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:51876)
0929 15:54:24.583958 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.584068 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260929 15:54:24.584470 29749 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:40773: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
I20260929 15:54:24.633868 29749 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } attempt: 1
I20260929 15:54:24.635179 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.634591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:51882)
0929 15:54:24.634820 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.634958 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260929 15:54:24.636231 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.635843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:51886)
0929 15:54:24.636023 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.636106 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260929 15:54:24.636454 29749 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:40773: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
I20260929 15:54:24.704706 29749 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } attempt: 2
I20260929 15:54:24.705895 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.705413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:51900)
0929 15:54:24.705644 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.705741 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260929 15:54:24.706914 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.706576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:51912)
0929 15:54:24.706702 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.706777 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260929 15:54:24.707160 29749 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:40773: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
I20260929 15:54:24.731397 29675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } attempt: 5
Sep 29 15:54:24 dist-test-slave-c51v krb5kdc[29595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697263, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for kudu/127.15.196.253@KRBTEST.COM
I20260929 15:54:24.746649 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.732147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:33725 (local address 127.0.0.1:49808)
0929 15:54:24.732373 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.732396 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:24.732540 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:24.733007 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:24.733010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:24.733367 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:24.733543 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:24.733551 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:24.734923 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:24.734927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:24.735886 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:24.735898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:24.736893 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:24.736897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:24.737028 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:24.737670 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:24.737688 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:24.740041 (+ 2353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:24.744410 (+ 4369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:24.744414 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:24.744417 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:24.744804 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:24.745209 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:24.745213 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:24.745215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:24.745336 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:24.745953 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:24.745956 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:24.746083 (+ 127us) client_negotiation.cc:769] Sending connection context
0929 15:54:24.746190 (+ 107us) client_negotiation.cc:240] Negotiation successful
0929 15:54:24.746395 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260929 15:54:24.746898 29754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.732337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49808 (local address 127.15.196.253:33725)
0929 15:54:24.732607 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:24.732617 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:24.732643 (+ 26us) server_negotiation.cc:407] Connection header received
0929 15:54:24.732718 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:24.732725 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:24.732842 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:24.732988 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:24.733713 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:24.734777 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:24.736046 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:24.736747 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:24.736868 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:24.740258 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:24.740266 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:24.740274 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:24.740324 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:24.744260 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:24.744919 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:24.744924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:24.744927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:24.745090 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:24.745434 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:24.745439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:24.745441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:24.745834 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:24.745997 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:24.746326 (+ 329us) server_negotiation.cc:299] Negotiation successful
0929 15:54:24.746583 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260929 15:54:24.747972 29675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } has no permanent_uuid. Determining permanent_uuid...
I20260929 15:54:24.748950 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.748595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:51922)
0929 15:54:24.748752 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.748830 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260929 15:54:24.749799 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.749474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:51926)
0929 15:54:24.749628 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.749703 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260929 15:54:24.749995 29675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:40773: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
I20260929 15:54:24.798370 29749 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } attempt: 3
I20260929 15:54:24.799633 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.799094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:51940)
0929 15:54:24.799363 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.799461 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260929 15:54:24.800688 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.800269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:51952)
0929 15:54:24.800493 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.800575 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260929 15:54:24.800877 29749 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:40773: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
I20260929 15:54:24.801199 29675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } attempt: 1
I20260929 15:54:24.802139 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.801737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:51966)
0929 15:54:24.801906 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.802000 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 15:54:24.803030 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.802717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:51982)
0929 15:54:24.802854 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.802925 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 15:54:24.803228 29675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:40773: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
I20260929 15:54:24.871469 29675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } attempt: 2
I20260929 15:54:24.872591 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.872124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:51994)
0929 15:54:24.872331 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.872439 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260929 15:54:24.873703 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.873304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:51998)
0929 15:54:24.873462 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.873548 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 15:54:24.873914 29675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:40773: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
W20260929 15:54:24.918401 29756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:24.918687 29756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:24.918838 29756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:24.926143 29756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:54:24.926267 29756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:24.926357 29756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:24.926451 29756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:54:24.926514 29756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:54:24.936866 29756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:35137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:38137,127.15.196.253:33725,127.15.196.252:40773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.252.keytab
--principal=kudu/127.15.196.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.252:40773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.196.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:24.938218 29756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:24.939839 29756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 15:54:24.950583 29749 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } attempt: 4
I20260929 15:54:24.952174 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.951421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:52012)
0929 15:54:24.951768 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.951921 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260929 15:54:24.953405 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.952964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:52022)
0929 15:54:24.953177 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.953264 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260929 15:54:24.953769 29749 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:40773: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
W20260929 15:54:24.961033 29762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:24.961242 29764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:24.961500 29761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:24.965019 29756 server_base.cc:1034] running on GCE node
I20260929 15:54:24.965907 29756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:24.967366 29756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:24.968613 29756 hybrid_clock.cc:648] HybridClock initialized: now 1790697264968559 us; error 56 us; skew 500 ppm
I20260929 15:54:24.968966 29756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:24 dist-test-slave-c51v krb5kdc[29595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:24.972173 29756 init.cc:377] Logged in from keytab as kudu/127.15.196.252@KRBTEST.COM (short username kudu)
I20260929 15:54:24.973654 29756 webserver.cc:460] Webserver started at http://127.15.196.252:36181/ using document root <none> and password file <none>
I20260929 15:54:24.974321 29756 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:24.974421 29756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:24.974674 29756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:54:24.977777 29756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/data/instance:
uuid: "105daf589838476d84c34bde2d49f553"
format_stamp: "Formatted at 2026-09-29 15:54:24 on dist-test-slave-c51v"
server_key: "36f277d8887cebf7aa68d5cb93261d0d"
server_key_iv: "d9d20096f72647e2cc58a78e5297ecc2"
server_key_version: "encryptionkey@0"
I20260929 15:54:24.978572 29756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/wal/instance:
uuid: "105daf589838476d84c34bde2d49f553"
format_stamp: "Formatted at 2026-09-29 15:54:24 on dist-test-slave-c51v"
server_key: "36f277d8887cebf7aa68d5cb93261d0d"
server_key_iv: "d9d20096f72647e2cc58a78e5297ecc2"
server_key_version: "encryptionkey@0"
I20260929 15:54:24.980129 29675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } attempt: 3
I20260929 15:54:24.981314 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.980751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:52026)
0929 15:54:24.981000 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.981119 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260929 15:54:24.982383 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:24.981946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:52030)
0929 15:54:24.982162 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:24.982253 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260929 15:54:24.982626 29675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:40773: Network error: Client connection negotiation failed: client connection to 127.15.196.252:40773: connect: Connection refused (error 111)
I20260929 15:54:24.984747 29756 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260929 15:54:24.988379 29771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:24.989714 29756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:54:24.989843 29756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/wal
uuid: "105daf589838476d84c34bde2d49f553"
format_stamp: "Formatted at 2026-09-29 15:54:24 on dist-test-slave-c51v"
server_key: "36f277d8887cebf7aa68d5cb93261d0d"
server_key_iv: "d9d20096f72647e2cc58a78e5297ecc2"
server_key_version: "encryptionkey@0"
I20260929 15:54:24.990000 29756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:25.025424 29756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:25.030659 29756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:25.030951 29756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:25.049965 29756 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.252:40773
I20260929 15:54:25.049995 29822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.252:40773 every 8 connection(s)
I20260929 15:54:25.051442 29756 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/data/info.pb
I20260929 15:54:25.051723 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 29756
I20260929 15:54:25.051918 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697059131395-16147-0/minicluster-data/master-2/wal/instance
I20260929 15:54:25.052477 16147 external_mini_cluster.cc:1428] Setting key 1cd85df2a256c1dd8042ffe1b90c3727
I20260929 15:54:25.056241 29823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:25.063830 29823 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:54:25 dist-test-slave-c51v krb5kdc[29595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.252@KRBTEST.COM
Sep 29 15:54:25 dist-test-slave-c51v krb5kdc[29595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.252@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:25.077191 29826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:25.054446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52036 (local address 127.15.196.252:40773)
0929 15:54:25.055217 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:25.055230 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:25.055271 (+ 41us) server_negotiation.cc:407] Connection header received
0929 15:54:25.057431 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:25.057456 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:25.057904 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:25.058372 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:25.060148 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.061715 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.063216 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.064041 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.064264 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:25.069402 (+ 5138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:25.069410 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:25.069421 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:25.069455 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:25.072694 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.073381 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.073388 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.073394 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.073558 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.073952 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.073957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.073960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.074516 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:25.074830 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:25.075281 (+ 451us) server_negotiation.cc:299] Negotiation successful
0929 15:54:25.075877 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":338,"threads_started":1}
I20260929 15:54:25.083262 29828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:25.066668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35206 (local address 127.15.196.254:38137)
0929 15:54:25.068213 (+ 1545us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:25.068219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:25.068241 (+ 22us) server_negotiation.cc:407] Connection header received
0929 15:54:25.068324 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:25.068329 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:25.068437 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:25.068546 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:25.070087 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.071268 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.072981 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.073446 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.073604 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:25.077188 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:25.077201 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:25.077206 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:25.077279 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:25.080475 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.081181 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.081187 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.081194 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.081350 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.081705 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.081710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.081714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.081991 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:25.082159 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:25.082618 (+ 459us) server_negotiation.cc:299] Negotiation successful
0929 15:54:25.082824 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1408,"thread_start_us":1278,"threads_started":1}
I20260929 15:54:25.083261 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:25.066591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38137 (local address 127.0.0.1:35206)
0929 15:54:25.067399 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:25.067451 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:25.067608 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:25.068892 (+ 1284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:25.068902 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:25.069542 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:25.069891 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:25.069901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.071599 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.071603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:25.072790 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:25.072803 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.073669 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.073676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:25.073864 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:25.074564 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:25.074582 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:25.076998 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:25.080662 (+ 3664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:25.080678 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:25.080683 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:25.081060 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:25.081462 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:25.081466 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:25.081470 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:25.081591 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:25.082121 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:25.082129 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:25.082319 (+ 190us) client_negotiation.cc:769] Sending connection context
0929 15:54:25.082514 (+ 195us) client_negotiation.cc:240] Negotiation successful
0929 15:54:25.082823 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":313,"threads_started":1}
I20260929 15:54:25.084748 29823 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:54:25 dist-test-slave-c51v krb5kdc[29595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
Sep 29 15:54:25 dist-test-slave-c51v krb5kdc[29595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.252@KRBTEST.COM for kudu/127.15.196.253@KRBTEST.COM
I20260929 15:54:25.096475 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:25.080182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35218 (local address 127.15.196.254:38137)
0929 15:54:25.080788 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:25.080792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:25.080809 (+ 17us) server_negotiation.cc:407] Connection header received
0929 15:54:25.080846 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:25.080849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:25.080900 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:25.081011 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:25.082225 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.083068 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.084604 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.085241 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.085388 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:25.090562 (+ 5174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:25.090568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:25.090571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:25.090600 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:25.093715 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.094424 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.094429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.094433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.094608 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.094986 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.094993 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.094996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.095265 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:25.095580 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:25.095839 (+ 259us) server_negotiation.cc:299] Negotiation successful
0929 15:54:25.096058 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":337,"threads_started":1}
I20260929 15:54:25.099408 29754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:25.085526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49814 (local address 127.15.196.253:33725)
0929 15:54:25.085689 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:25.085695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:25.085875 (+ 180us) server_negotiation.cc:407] Connection header received
0929 15:54:25.086055 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:25.086059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:25.086121 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:25.086195 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:25.086969 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.088120 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.089519 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.089933 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.090035 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:25.093690 (+ 3655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:25.093696 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:25.093700 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:25.093738 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:25.096474 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.097246 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.097251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.097254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.097410 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.097875 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.097882 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.097885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.098260 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:25.098427 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:25.098915 (+ 488us) server_negotiation.cc:299] Negotiation successful
0929 15:54:25.099074 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 15:54:25.099737 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:25.085529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:33725 (local address 127.0.0.1:49814)
0929 15:54:25.085804 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:25.085822 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:25.085927 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:25.086252 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:25.086260 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:25.086664 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:25.086808 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:25.086816 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.088259 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.088262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:25.089338 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:25.089353 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.090101 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.090108 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:25.090344 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:25.090923 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:25.090939 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:25.093466 (+ 2527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:25.096596 (+ 3130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:25.096601 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:25.096605 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:25.097121 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:25.097620 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:25.097623 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:25.097626 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:25.097760 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:25.098398 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:25.098405 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:25.098631 (+ 226us) client_negotiation.cc:769] Sending connection context
0929 15:54:25.098824 (+ 193us) client_negotiation.cc:240] Negotiation successful
0929 15:54:25.099193 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
Sep 29 15:54:25 dist-test-slave-c51v krb5kdc[29595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.253@KRBTEST.COM
I20260929 15:54:25.108600 29823 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } has no permanent_uuid. Determining permanent_uuid...
I20260929 15:54:25.115456 29754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:25.098232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49824 (local address 127.15.196.253:33725)
0929 15:54:25.099654 (+ 1422us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:25.099660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:25.099676 (+ 16us) server_negotiation.cc:407] Connection header received
0929 15:54:25.099719 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:25.099724 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:25.099790 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:25.099911 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:25.101245 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.102755 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.104067 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.105000 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.105156 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:25.109682 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:25.109687 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:25.109690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:25.109744 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:25.112705 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.113416 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.113422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.113425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.113633 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.114033 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.114038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.114041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.114279 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:25.114587 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:25.114846 (+ 259us) server_negotiation.cc:299] Negotiation successful
0929 15:54:25.115060 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1335,"thread_start_us":9261,"threads_started":1}
Sep 29 15:54:25 dist-test-slave-c51v krb5kdc[29595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.252@KRBTEST.COM for kudu/127.15.196.252@KRBTEST.COM
I20260929 15:54:25.120724 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:54:25.123140 29826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:25.109414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52046 (local address 127.15.196.252:40773)
0929 15:54:25.109579 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:25.109586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:25.109605 (+ 19us) server_negotiation.cc:407] Connection header received
0929 15:54:25.109775 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:25.109783 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:25.109860 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:25.109958 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:25.110741 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.111880 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.113083 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.113565 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.113699 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:25.116905 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:25.116909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:25.116913 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:25.116939 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:25.120036 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.120938 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.120946 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.120952 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.121091 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.121539 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.121547 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.121551 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.121830 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:25.122014 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:25.122385 (+ 371us) server_negotiation.cc:299] Negotiation successful
0929 15:54:25.122673 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":47}
I20260929 15:54:25.123257 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:25.109353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:52046)
0929 15:54:25.109546 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:25.109567 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:25.109667 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:25.109983 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:25.109986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:25.110386 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:25.110607 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:25.110617 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.112018 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.112022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:25.112939 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:25.112955 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.113749 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.113755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:25.113952 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:25.114554 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:25.114574 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:25.116728 (+ 2154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:25.120184 (+ 3456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:25.120190 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:25.120194 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:25.120825 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:25.121211 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:25.121219 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:25.121225 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:25.121424 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:25.121972 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:25.121977 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:25.122125 (+ 148us) client_negotiation.cc:769] Sending connection context
0929 15:54:25.122289 (+ 164us) client_negotiation.cc:240] Negotiation successful
0929 15:54:25.122844 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":199}
I20260929 15:54:25.130268 29823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553: Bootstrap starting.
I20260929 15:54:25.134611 29823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:25.136193 29823 log.cc:824] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:25.143585 29823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553: No bootstrap required, opened a new log
I20260929 15:54:25.148017 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:25.126345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35222 (local address 127.15.196.254:38137)
0929 15:54:25.126537 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:25.126543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:25.128456 (+ 1913us) server_negotiation.cc:407] Connection header received
0929 15:54:25.128728 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:25.128733 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:25.128832 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:25.128899 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:25.131368 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.132495 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.134827 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.135485 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.135579 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:25.140520 (+ 4941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:25.140528 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:25.140534 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:25.140582 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:25.144593 (+ 4011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.145506 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.145511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.145514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.145667 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.146155 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.146160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.146163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.146499 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:25.146729 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:25.147382 (+ 653us) server_negotiation.cc:299] Negotiation successful
0929 15:54:25.147684 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 15:54:25.149192 29754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:25.127035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49830 (local address 127.15.196.253:33725)
0929 15:54:25.127213 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:25.127218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:25.127805 (+ 587us) server_negotiation.cc:407] Connection header received
0929 15:54:25.128009 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:25.128019 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:25.128123 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:25.128251 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:25.131096 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.132678 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.135837 (+ 3159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.136776 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.136914 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:25.141279 (+ 4365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:25.141287 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:25.141292 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:25.141337 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:25.145084 (+ 3747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.146230 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.146239 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.146244 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.146474 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.147051 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.147058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.147061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.147527 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:25.147805 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:25.148459 (+ 654us) server_negotiation.cc:299] Negotiation successful
0929 15:54:25.148774 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 15:54:25.154107 29823 raft_consensus.cc:348] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } }
I20260929 15:54:25.154492 29823 raft_consensus.cc:374] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:25.154634 29823 raft_consensus.cc:729] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 105daf589838476d84c34bde2d49f553, State: Initialized, Role: FOLLOWER
I20260929 15:54:25.155325 29823 consensus_queue.cc:260] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } }
I20260929 15:54:25.156458 29839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } } }
I20260929 15:54:25.156948 29839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:54:25.157004 29826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:25.128955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52048 (local address 127.15.196.252:40773)
0929 15:54:25.130001 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:25.130009 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:25.137680 (+ 7671us) server_negotiation.cc:407] Connection header received
0929 15:54:25.137871 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:25.137880 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:25.137982 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:25.138127 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:25.140549 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.141876 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.144189 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.145033 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.145156 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:25.150093 (+ 4937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:25.150111 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:25.150115 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:25.150151 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:25.153745 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.154463 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.154468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.154473 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.154672 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.155143 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.155148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.155152 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.155560 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:25.155877 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:25.156248 (+ 371us) server_negotiation.cc:299] Negotiation successful
0929 15:54:25.156584 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":877}
I20260929 15:54:25.157617 29823 sys_catalog.cc:564] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:54:25.159849 29675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } attempt: 4
Sep 29 15:54:25 dist-test-slave-c51v krb5kdc[29595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697263, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for kudu/127.15.196.252@KRBTEST.COM
W20260929 15:54:25.172195 29850 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 15:54:25.172353 29850 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 15:54:25.177732 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:25.160499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:52058)
0929 15:54:25.160703 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:25.160725 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:25.160851 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:25.161238 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:25.161244 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:25.161792 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:25.162024 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:25.162041 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.163573 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.163579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:25.164795 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:25.164814 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.165814 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.165820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:25.166034 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:25.166754 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:25.166787 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:25.169662 (+ 2875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:25.174624 (+ 4962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:25.174630 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:25.174634 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:25.175168 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:25.175681 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:25.175688 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:25.175693 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:25.175853 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:25.176699 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:25.176705 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:25.176873 (+ 168us) client_negotiation.cc:769] Sending connection context
0929 15:54:25.177011 (+ 138us) client_negotiation.cc:240] Negotiation successful
0929 15:54:25.177322 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260929 15:54:25.166891 29756 master_runner.cc:186] Error getting master registration for 127.15.196.254:38137: OK, instance_id { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" instance_seqno: 1790697264014878 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 15:54:25.178685 29756 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 15:54:25.179867 29826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:25.160586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52058 (local address 127.15.196.252:40773)
0929 15:54:25.160750 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:25.160759 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:25.160788 (+ 29us) server_negotiation.cc:407] Connection header received
0929 15:54:25.160981 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:25.160986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:25.161095 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:25.161213 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:25.162202 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.163395 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.164934 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.165647 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.165789 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:25.169898 (+ 4109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:25.169904 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:25.169909 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:25.169944 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:25.174450 (+ 4506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.175371 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.175376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.175380 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.175531 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.176053 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.176060 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.176064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.176540 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:25.179059 (+ 2519us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:25.179184 (+ 125us) server_negotiation.cc:299] Negotiation successful
0929 15:54:25.179443 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 15:54:25.184628 29675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e: Bootstrap starting.
I20260929 15:54:25.188303 29675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:25.190428 29675 log.cc:824] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:25.193707 29675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e: No bootstrap required, opened a new log
I20260929 15:54:25.204707 29675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } }
I20260929 15:54:25.205123 29675 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:25.205245 29675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d9ab46751154c709e7d9d48be4dd66e, State: Initialized, Role: FOLLOWER
I20260929 15:54:25.205940 29675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } }
I20260929 15:54:25.206871 29853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } } }
I20260929 15:54:25.207237 29853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:54:25.208361 29675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e [sys.catalog]: configured and running, proceeding with master startup.
W20260929 15:54:25.218393 29864 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 15:54:25.219431 29864 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 15:54:25.217670 29608 master_runner.cc:186] Error getting master registration for 127.15.196.253:33725: OK, instance_id { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" instance_seqno: 1790697264514317 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 15:54:25.228997 29608 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 15:54:25.253012 29749 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } attempt: 5
W20260929 15:54:25.254253 29629 tablet.cc:2369] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 15:54:25 dist-test-slave-c51v krb5kdc[29595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697264, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.253@KRBTEST.COM for kudu/127.15.196.252@KRBTEST.COM
I20260929 15:54:25.270303 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:25.253639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:40773 (local address 127.0.0.1:52074)
0929 15:54:25.253905 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:25.253936 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:25.254090 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:25.254667 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:25.254673 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:25.255203 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:25.255474 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:25.255489 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.256937 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.256943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:25.258434 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:25.258450 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.259692 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.259698 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:25.259919 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:25.260607 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:25.260633 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:25.263443 (+ 2810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:25.267505 (+ 4062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:25.267511 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:25.267515 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:25.268006 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:25.268448 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:25.268453 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:25.268457 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:25.268649 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:25.269274 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:25.269279 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:25.269444 (+ 165us) client_negotiation.cc:769] Sending connection context
0929 15:54:25.269601 (+ 157us) client_negotiation.cc:240] Negotiation successful
0929 15:54:25.269889 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260929 15:54:25.270674 29826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:25.253734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52074 (local address 127.15.196.252:40773)
0929 15:54:25.253903 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:25.253916 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:25.254222 (+ 306us) server_negotiation.cc:407] Connection header received
0929 15:54:25.254287 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:25.254293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:25.254382 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:25.254478 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:25.255631 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.256806 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.258633 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:25.259369 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:25.259492 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:25.263629 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:25.263636 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:25.263641 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:25.263680 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:25.267353 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.268147 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.268157 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.268161 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.268327 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:25.268774 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:25.268779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:25.268783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:25.269136 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:25.269305 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:25.269713 (+ 408us) server_negotiation.cc:299] Negotiation successful
0929 15:54:25.269962 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 15:54:25.276706 29749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db: Bootstrap starting.
I20260929 15:54:25.280918 29749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:25.282224 29749 log.cc:824] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db: Log is configured to *not* fsync() on all Append() calls
W20260929 15:54:25.283068 29777 tablet.cc:2369] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:54:25.285354 29749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db: No bootstrap required, opened a new log
I20260929 15:54:25.293180 29749 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } }
I20260929 15:54:25.293741 29749 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:25.293864 29749 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0deb7499d933420f99eebcf8aef865db, State: Initialized, Role: FOLLOWER
I20260929 15:54:25.295799 29749 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } }
I20260929 15:54:25.296795 29867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } } }
I20260929 15:54:25.297247 29867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:54:25.298157 29749 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:54:25.316140 29682 master_runner.cc:263] No leader master found from master 0deb7499d933420f99eebcf8aef865db
I20260929 15:54:25.316300 29682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 15:54:25.318245 29878 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 15:54:25.318394 29878 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 15:54:25.352283 29839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:54:25.352629 29839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } }
I20260929 15:54:25.354274 29839 leader_election.cc:288] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d9ab46751154c709e7d9d48be4dd66e (127.15.196.254:38137), 0deb7499d933420f99eebcf8aef865db (127.15.196.253:33725)
I20260929 15:54:25.355451 29650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "105daf589838476d84c34bde2d49f553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d9ab46751154c709e7d9d48be4dd66e" is_pre_election: true
I20260929 15:54:25.356091 29650 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 105daf589838476d84c34bde2d49f553 in term 0.
I20260929 15:54:25.356714 29774 leader_election.cc:302] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 105daf589838476d84c34bde2d49f553, 6d9ab46751154c709e7d9d48be4dd66e; no voters:
I20260929 15:54:25.357651 29724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "105daf589838476d84c34bde2d49f553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0deb7499d933420f99eebcf8aef865db" is_pre_election: true
I20260929 15:54:25.358290 29724 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 105daf589838476d84c34bde2d49f553 in term 0.
I20260929 15:54:25.359139 29839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:54:25.359315 29839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:54:25.359423 29839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:25.362771 29839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } }
I20260929 15:54:25.363895 29650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "105daf589838476d84c34bde2d49f553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d9ab46751154c709e7d9d48be4dd66e"
I20260929 15:54:25.364135 29650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:25.367693 29650 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 105daf589838476d84c34bde2d49f553 in term 1.
I20260929 15:54:25.368121 29774 leader_election.cc:302] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 105daf589838476d84c34bde2d49f553, 6d9ab46751154c709e7d9d48be4dd66e; no voters:
I20260929 15:54:25.370550 29724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "105daf589838476d84c34bde2d49f553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0deb7499d933420f99eebcf8aef865db"
I20260929 15:54:25.370787 29724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:25.374260 29724 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 105daf589838476d84c34bde2d49f553 in term 1.
I20260929 15:54:25.374997 29839 leader_election.cc:288] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [CANDIDATE]: Term 1 election: Requested vote from peers 6d9ab46751154c709e7d9d48be4dd66e (127.15.196.254:38137), 0deb7499d933420f99eebcf8aef865db (127.15.196.253:33725)
I20260929 15:54:25.375212 29839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:54:25.375360 29839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [term 1 LEADER]: Becoming Leader. State: Replica: 105daf589838476d84c34bde2d49f553, State: Running, Role: LEADER
I20260929 15:54:25.376675 29839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } }
I20260929 15:54:25.394002 29880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 105daf589838476d84c34bde2d49f553. Latest consensus state: current_term: 1 leader_uuid: "105daf589838476d84c34bde2d49f553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } } }
I20260929 15:54:25.394639 29880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:25.401680 29882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:54:25.404062 29882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:54:25.424281 29724 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db [term 1 FOLLOWER]: Refusing update from remote peer 105daf589838476d84c34bde2d49f553: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:54:25.425311 29880 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:54:25.432677 29650 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e [term 1 FOLLOWER]: Refusing update from remote peer 105daf589838476d84c34bde2d49f553: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:54:25.433655 29880 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:54:25.458359 29867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 105daf589838476d84c34bde2d49f553. Latest consensus state: current_term: 1 leader_uuid: "105daf589838476d84c34bde2d49f553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } } }
I20260929 15:54:25.458690 29867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:54:25.471797 29867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "105daf589838476d84c34bde2d49f553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } } }
I20260929 15:54:25.472069 29867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:54:25.473028 29883 mvcc.cc:205] Tried to move back new op lower bound from 7334695999164575744 to 7334695998994022400. Current Snapshot: MvccSnapshot[applied={T|T < 7334695999164575744}]
I20260929 15:54:25.479739 29839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "105daf589838476d84c34bde2d49f553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } } }
I20260929 15:54:25.480042 29839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:25.486331 29882 catalog_manager.cc:1347] Generated new cluster ID: 3e1578e57bc64308935820cb5883a401
I20260929 15:54:25.486444 29882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:54:25.505524 29882 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:54:25.507553 29882 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:54:25.510317 29853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 105daf589838476d84c34bde2d49f553. Latest consensus state: current_term: 1 leader_uuid: "105daf589838476d84c34bde2d49f553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } } }
I20260929 15:54:25.510643 29853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:54:25.547380 29885 mvcc.cc:205] Tried to move back new op lower bound from 7334695999164575744 to 7334695998994022400. Current Snapshot: MvccSnapshot[applied={T|T < 7334695999164575744}]
I20260929 15:54:25.532250 29880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "105daf589838476d84c34bde2d49f553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } } }
I20260929 15:54:25.554817 29880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553 [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:25.557544 29853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "105daf589838476d84c34bde2d49f553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d9ab46751154c709e7d9d48be4dd66e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38137 } } peers { permanent_uuid: "0deb7499d933420f99eebcf8aef865db" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 33725 } } peers { permanent_uuid: "105daf589838476d84c34bde2d49f553" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 40773 } } }
I20260929 15:54:25.557817 29853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:54:25.579095 29882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 105daf589838476d84c34bde2d49f553: Generated new TSK 0
I20260929 15:54:25.580371 29882 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:54:26.221583 29864 catalog_manager.cc:1259] Loaded cluster ID: 3e1578e57bc64308935820cb5883a401
I20260929 15:54:26.221688 29864 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e: loading cluster ID for follower catalog manager: success
I20260929 15:54:26.224370 29864 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e: acquiring CA information for follower catalog manager: success
I20260929 15:54:26.226145 29864 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6d9ab46751154c709e7d9d48be4dd66e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 15:54:26.320358 29878 catalog_manager.cc:1259] Loaded cluster ID: 3e1578e57bc64308935820cb5883a401
I20260929 15:54:26.320503 29878 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db: loading cluster ID for follower catalog manager: success
I20260929 15:54:26.322962 29878 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db: acquiring CA information for follower catalog manager: success
I20260929 15:54:26.324827 29878 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0deb7499d933420f99eebcf8aef865db: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 15:54:26.713049 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 29608
I20260929 15:54:26.714322 29670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:26.828842 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 29608
I20260929 15:54:26.850121 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 29682
I20260929 15:54:26.851361 29744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:26.960897 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 29682
I20260929 15:54:26.981299 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 29756
I20260929 15:54:26.982401 29818 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:27.091899 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 29756
2026-09-29T15:54:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4834 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:54:27 dist-test-slave-c51v krb5kdc[29903](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:54:27 dist-test-slave-c51v krb5kdc[29903](info): set up 2 sockets
Sep 29 15:54:27 dist-test-slave-c51v krb5kdc[29903](info): commencing operation
krb5kdc: starting...
W20260929 15:54:28.193629 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:54:28 dist-test-slave-c51v krb5kdc[29903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:54:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:54:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:54:28.432070 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:40177
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:41479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:40177
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:54:28.762274 29916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:28.762629 29916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:28.762763 29916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:28.769979 29916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:54:28.770087 29916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:28.770133 29916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:28.770218 29916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:54:28.770296 29916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:54:28.780411 29916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:41479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:40177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:40177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:28.781687 29916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:28.782940 29916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:28.794760 29921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:28.795699 29924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:28.795902 29922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:28.797068 29916 server_base.cc:1034] running on GCE node
I20260929 15:54:28.797825 29916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:28.799265 29916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:28.800576 29916 hybrid_clock.cc:648] HybridClock initialized: now 1790697268800529 us; error 50 us; skew 500 ppm
I20260929 15:54:28.800952 29916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:28 dist-test-slave-c51v krb5kdc[29903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697268, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:28.809919 29916 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:54:28.811673 29916 webserver.cc:460] Webserver started at http://127.15.196.254:40915/ using document root <none> and password file <none>
I20260929 15:54:28.812445 29916 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:28.812602 29916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:28.812927 29916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:54:28.816078 29916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "b824f261d9db42949a08258407961a77"
format_stamp: "Formatted at 2026-09-29 15:54:28 on dist-test-slave-c51v"
server_key: "14622a0f4bc3be063dc9a72ede94f024"
server_key_iv: "c69c017fc295710447f4e34f91d21e5d"
server_key_version: "encryptionkey@0"
I20260929 15:54:28.816876 29916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "b824f261d9db42949a08258407961a77"
format_stamp: "Formatted at 2026-09-29 15:54:28 on dist-test-slave-c51v"
server_key: "14622a0f4bc3be063dc9a72ede94f024"
server_key_iv: "c69c017fc295710447f4e34f91d21e5d"
server_key_version: "encryptionkey@0"
I20260929 15:54:28.822197 29916 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 15:54:28.825734 29931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:28.827075 29916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 15:54:28.827268 29916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "b824f261d9db42949a08258407961a77"
format_stamp: "Formatted at 2026-09-29 15:54:28 on dist-test-slave-c51v"
server_key: "14622a0f4bc3be063dc9a72ede94f024"
server_key_iv: "c69c017fc295710447f4e34f91d21e5d"
server_key_version: "encryptionkey@0"
I20260929 15:54:28.827539 29916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:28.852823 29916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:28.856938 29916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:28.857254 29916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:28.875887 29916 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:40177
I20260929 15:54:28.875910 29982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:40177 every 8 connection(s)
I20260929 15:54:28.877352 29916 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:54:28.881156 29983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:28.885149 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 29916
I20260929 15:54:28.885387 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:54:28.886194 16147 external_mini_cluster.cc:1428] Setting key 3e48002561e9942c17e38d04f4beda0e
I20260929 15:54:28.894886 29983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77: Bootstrap starting.
I20260929 15:54:28.899597 29983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:28.901149 29983 log.cc:824] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77: Log is configured to *not* fsync() on all Append() calls
Sep 29 15:54:28 dist-test-slave-c51v krb5kdc[29903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:28.904559 29983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77: No bootstrap required, opened a new log
I20260929 15:54:28.910635 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:28.888151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41192 (local address 127.15.196.254:40177)
0929 15:54:28.888959 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:28.888981 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:28.889030 (+ 49us) server_negotiation.cc:407] Connection header received
0929 15:54:28.890657 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:28.890679 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:28.891107 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:28.891504 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:28.893342 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:28.894630 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:28.896493 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:28.897151 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:28.897281 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:28.903176 (+ 5895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:28.903184 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:28.903199 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:28.903233 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:28.906396 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:28.907356 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:28.907362 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:28.907368 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:28.907554 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:28.907960 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:28.907963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:28.907965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:28.908477 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:28.908686 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:28.909162 (+ 476us) server_negotiation.cc:299] Negotiation successful
0929 15:54:28.909535 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":390,"threads_started":1}
I20260929 15:54:28.914124 29983 raft_consensus.cc:348] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b824f261d9db42949a08258407961a77" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40177 } }
I20260929 15:54:28.914516 29983 raft_consensus.cc:374] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:28.914644 29983 raft_consensus.cc:729] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b824f261d9db42949a08258407961a77, State: Initialized, Role: FOLLOWER
I20260929 15:54:28.914705 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:54:28.915252 29983 consensus_queue.cc:260] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b824f261d9db42949a08258407961a77" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40177 } }
I20260929 15:54:28.915584 29983 raft_consensus.cc:388] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:54:28.915717 29983 raft_consensus.cc:482] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:54:28.915870 29983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:28.918788 29983 raft_consensus.cc:504] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b824f261d9db42949a08258407961a77" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40177 } }
I20260929 15:54:28.919236 29983 leader_election.cc:302] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b824f261d9db42949a08258407961a77; no voters:
I20260929 15:54:28.919842 29983 leader_election.cc:288] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:54:28.919991 29988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:54:28.920732 29988 raft_consensus.cc:686] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [term 1 LEADER]: Becoming Leader. State: Replica: b824f261d9db42949a08258407961a77, State: Running, Role: LEADER
I20260929 15:54:28.921299 29988 consensus_queue.cc:237] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b824f261d9db42949a08258407961a77" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40177 } }
I20260929 15:54:28.921932 29983 sys_catalog.cc:564] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:54:28.924167 29989 sys_catalog.cc:455] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b824f261d9db42949a08258407961a77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b824f261d9db42949a08258407961a77" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40177 } } }
I20260929 15:54:28.924651 29989 sys_catalog.cc:458] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:28.926225 29990 sys_catalog.cc:455] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b824f261d9db42949a08258407961a77. Latest consensus state: current_term: 1 leader_uuid: "b824f261d9db42949a08258407961a77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b824f261d9db42949a08258407961a77" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 40177 } } }
I20260929 15:54:28.926491 29990 sys_catalog.cc:458] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77 [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:28.927461 29995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:54:28.933147 29995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:54:28.942682 29995 catalog_manager.cc:1347] Generated new cluster ID: b5900c9902b347ab88af7ff198284288
I20260929 15:54:28.942817 29995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:54:28.951599 29995 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:54:28.953428 29995 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:54:28.960073 29995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b824f261d9db42949a08258407961a77: Generated new TSK 0
I20260929 15:54:28.960870 29995 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:54:28.975944 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 29916
I20260929 15:54:28.977137 29978 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:29.067247 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 29916
2026-09-29T15:54:29Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1970 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:54:29 dist-test-slave-c51v krb5kdc[30011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:54:29 dist-test-slave-c51v krb5kdc[30011](info): set up 2 sockets
Sep 29 15:54:29 dist-test-slave-c51v krb5kdc[30011](info): commencing operation
krb5kdc: starting...
W20260929 15:54:30.165712 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:54:30 dist-test-slave-c51v krb5kdc[30011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:54:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:54:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:54:30.406764 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:44615
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:35523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:44615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:54:30.737946 30024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:30.738273 30024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:30.738461 30024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:30.745743 30024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:54:30.745857 30024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:30.745903 30024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:30.745987 30024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:54:30.746060 30024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:54:30.756274 30024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:35523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:44615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:44615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:30.757725 30024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:30.759102 30024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:30.771353 30032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:30.771199 30030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:30.771564 30029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:30.771736 30024 server_base.cc:1034] running on GCE node
I20260929 15:54:30.772783 30024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:30.774209 30024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:30.775416 30024 hybrid_clock.cc:648] HybridClock initialized: now 1790697270775377 us; error 34 us; skew 500 ppm
I20260929 15:54:30.775741 30024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:30 dist-test-slave-c51v krb5kdc[30011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697270, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:30.778828 30024 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:54:30.780443 30024 webserver.cc:460] Webserver started at http://127.15.196.254:44833/ using document root <none> and password file <none>
I20260929 15:54:30.781152 30024 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:30.781297 30024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:30.781613 30024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:54:30.784633 30024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "4d9bd3eddcd34685a24d0466f4879f6e"
format_stamp: "Formatted at 2026-09-29 15:54:30 on dist-test-slave-c51v"
server_key: "4e27288f0b67470df12cc7f7d0a2177b"
server_key_iv: "b565c6203ccd626c48ac0b46be5c832e"
server_key_version: "encryptionkey@0"
I20260929 15:54:30.785408 30024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "4d9bd3eddcd34685a24d0466f4879f6e"
format_stamp: "Formatted at 2026-09-29 15:54:30 on dist-test-slave-c51v"
server_key: "4e27288f0b67470df12cc7f7d0a2177b"
server_key_iv: "b565c6203ccd626c48ac0b46be5c832e"
server_key_version: "encryptionkey@0"
I20260929 15:54:30.790506 30024 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 15:54:30.793946 30039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:30.795522 30024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:54:30.795753 30024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "4d9bd3eddcd34685a24d0466f4879f6e"
format_stamp: "Formatted at 2026-09-29 15:54:30 on dist-test-slave-c51v"
server_key: "4e27288f0b67470df12cc7f7d0a2177b"
server_key_iv: "b565c6203ccd626c48ac0b46be5c832e"
server_key_version: "encryptionkey@0"
I20260929 15:54:30.795960 30024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:30.822762 30024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:30.827363 30024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:30.827666 30024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:30.846266 30024 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:44615
I20260929 15:54:30.846297 30090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:44615 every 8 connection(s)
I20260929 15:54:30.847694 30024 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:54:30.849548 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 30024
I20260929 15:54:30.849774 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:54:30.850636 16147 external_mini_cluster.cc:1428] Setting key 640d02a5214d6d27db06edddfa883d51
I20260929 15:54:30.852273 30091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:30.863552 30091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e: Bootstrap starting.
Sep 29 15:54:30 dist-test-slave-c51v krb5kdc[30011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:30.867187 30091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:30.868577 30091 log.cc:824] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:30.871991 30091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e: No bootstrap required, opened a new log
I20260929 15:54:30.873063 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:30.852749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44196 (local address 127.15.196.254:44615)
0929 15:54:30.853529 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:30.853541 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:30.853588 (+ 47us) server_negotiation.cc:407] Connection header received
0929 15:54:30.855145 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:30.855166 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:30.855633 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:30.856045 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:30.857523 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:30.858828 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:30.860255 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:30.860955 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:30.861173 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:30.865720 (+ 4547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:30.865727 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:30.865741 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:30.865777 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:30.868957 (+ 3180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:30.869746 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:30.869757 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:30.869769 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:30.869961 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:30.870376 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:30.870379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:30.870381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:30.870900 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:30.871128 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:30.871603 (+ 475us) server_negotiation.cc:299] Negotiation successful
0929 15:54:30.871994 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":335,"threads_started":1}
I20260929 15:54:30.879755 30091 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } }
I20260929 15:54:30.880204 30091 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:30.880313 30091 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9bd3eddcd34685a24d0466f4879f6e, State: Initialized, Role: FOLLOWER
I20260929 15:54:30.880887 30091 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } }
I20260929 15:54:30.881145 30091 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:54:30.881295 30091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:54:30.881481 30091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:30.883965 30091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } }
I20260929 15:54:30.884490 30091 leader_election.cc:302] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d9bd3eddcd34685a24d0466f4879f6e; no voters:
I20260929 15:54:30.885067 30091 leader_election.cc:288] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:54:30.885213 30096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:54:30.885876 30096 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 1 LEADER]: Becoming Leader. State: Replica: 4d9bd3eddcd34685a24d0466f4879f6e, State: Running, Role: LEADER
I20260929 15:54:30.886474 30096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } }
I20260929 15:54:30.887251 30091 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:54:30.889364 30097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } } }
I20260929 15:54:30.889842 30097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:30.890048 30098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d9bd3eddcd34685a24d0466f4879f6e. Latest consensus state: current_term: 1 leader_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } } }
I20260929 15:54:30.890475 30098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:30.892854 30104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:54:30.897795 30104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:54:30.906965 30104 catalog_manager.cc:1347] Generated new cluster ID: 8dfd4078f4b64f8cb4a2236da0e388ff
I20260929 15:54:30.907127 30104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:54:30.923511 30104 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:54:30.924871 30104 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:54:30.931828 30104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e: Generated new TSK 0
I20260929 15:54:30.932700 30104 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:54:30.999099 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44615
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:35523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:54:31.332360 30117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:31.332661 30117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:31.332756 30117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:31.340473 30117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:31.340627 30117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:31.340771 30117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:54:31.350378 30117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:35523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:31.351732 30117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:31.352981 30117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:31.365891 30122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:31.365921 30123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:31.365955 30125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:31.366845 30117 server_base.cc:1034] running on GCE node
I20260929 15:54:31.367496 30117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:31.368407 30117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:31.369604 30117 hybrid_clock.cc:648] HybridClock initialized: now 1790697271369559 us; error 33 us; skew 500 ppm
I20260929 15:54:31.369941 30117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:31 dist-test-slave-c51v krb5kdc[30011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697271, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:31.373219 30117 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:54:31.374660 30117 webserver.cc:460] Webserver started at http://127.15.196.193:38839/ using document root <none> and password file <none>
I20260929 15:54:31.375412 30117 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:31.375519 30117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:31.375761 30117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:54:31.378528 30117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "d24a657d04014827a741baae1dcd09a7"
format_stamp: "Formatted at 2026-09-29 15:54:31 on dist-test-slave-c51v"
server_key: "acd541a656e3765dbe63472ebc5629d4"
server_key_iv: "5489a1599fcc40057dbd102d43bfbe4b"
server_key_version: "encryptionkey@0"
I20260929 15:54:31.379235 30117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "d24a657d04014827a741baae1dcd09a7"
format_stamp: "Formatted at 2026-09-29 15:54:31 on dist-test-slave-c51v"
server_key: "acd541a656e3765dbe63472ebc5629d4"
server_key_iv: "5489a1599fcc40057dbd102d43bfbe4b"
server_key_version: "encryptionkey@0"
I20260929 15:54:31.384315 30117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 15:54:31.387615 30132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:31.388952 30117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:54:31.389130 30117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "d24a657d04014827a741baae1dcd09a7"
format_stamp: "Formatted at 2026-09-29 15:54:31 on dist-test-slave-c51v"
server_key: "acd541a656e3765dbe63472ebc5629d4"
server_key_iv: "5489a1599fcc40057dbd102d43bfbe4b"
server_key_version: "encryptionkey@0"
I20260929 15:54:31.389312 30117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:31.416564 30117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:31.421097 30117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:31.421398 30117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:31.422441 30117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:31.423851 30117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:31.423923 30117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:31.424079 30117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:31.424144 30117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:31.456839 30117 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:46253
I20260929 15:54:31.456882 30244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:46253 every 8 connection(s)
I20260929 15:54:31.458146 30117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:54:31.462278 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 30117
I20260929 15:54:31.462491 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:54:31.463197 16147 external_mini_cluster.cc:1428] Setting key 86ff6b8c7cc95c7794496d04967c03fe
Sep 29 15:54:31 dist-test-slave-c51v krb5kdc[30011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697271, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:31.478157 30247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:31.460708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:50459 (local address 127.15.196.254:44615)
0929 15:54:31.461134 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:31.461138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:31.462104 (+ 966us) server_negotiation.cc:407] Connection header received
0929 15:54:31.464156 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:31.464160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:31.464223 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:31.464344 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:31.465986 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:31.466672 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:31.468295 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:31.468699 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:31.468834 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:31.472449 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:31.472455 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:31.472458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:31.472491 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:31.475051 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:31.475832 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:31.475837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:31.475840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:31.475953 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:31.476381 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:31.476385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:31.476389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:31.476643 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:31.476798 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:31.477538 (+ 740us) server_negotiation.cc:299] Negotiation successful
0929 15:54:31.477764 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":206,"threads_started":1}
I20260929 15:54:31.479096 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:31.461269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:50459)
0929 15:54:31.461927 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:31.461965 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:31.463832 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:31.464597 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:31.464605 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:31.465263 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:31.465812 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:31.465829 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:31.466826 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:31.466830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:31.468140 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:31.468156 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:31.468935 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:31.468944 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:31.469177 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:31.469875 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:31.469896 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:31.472230 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:31.475187 (+ 2957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:31.475196 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:31.475213 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:31.475699 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:31.476072 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:31.476078 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:31.476081 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:31.476264 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:31.476770 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:31.476778 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:31.477144 (+ 366us) client_negotiation.cc:769] Sending connection context
0929 15:54:31.477410 (+ 266us) client_negotiation.cc:240] Negotiation successful
0929 15:54:31.477971 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":246,"threads_started":1}
I20260929 15:54:31.480837 30245 heartbeater.cc:347] Connected to a master server at 127.15.196.254:44615
I20260929 15:54:31.481199 30245 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:31.482189 30245 heartbeater.cc:511] Master 127.15.196.254:44615 requested a full tablet report, sending...
I20260929 15:54:31.484481 30056 ts_manager.cc:194] Registered new tserver with Master: d24a657d04014827a741baae1dcd09a7 (127.15.196.193:46253)
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
I20260929 15:54:31.486104 30056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:50459
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:54:31.514796 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44615
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:35523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:54:31.843807 30251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:31.844087 30251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:31.844242 30251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:31.855108 30251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:31.855255 30251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:31.855453 30251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:54:31.870909 30251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:35523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:31.872255 30251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:31.873632 30251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:31.887128 30256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:31.887328 30257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:31.887420 30259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:31.887928 30251 server_base.cc:1034] running on GCE node
I20260929 15:54:31.888631 30251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:31.889578 30251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:31.890774 30251 hybrid_clock.cc:648] HybridClock initialized: now 1790697271890723 us; error 39 us; skew 500 ppm
I20260929 15:54:31.891098 30251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:31 dist-test-slave-c51v krb5kdc[30011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697271, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:31.894490 30251 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:54:31.896008 30251 webserver.cc:460] Webserver started at http://127.15.196.194:43785/ using document root <none> and password file <none>
I20260929 15:54:31.896715 30251 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:31.896867 30251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:31.897197 30251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:54:31.900425 30251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "5f7986e698c643f69a2ab6aa83873ae0"
format_stamp: "Formatted at 2026-09-29 15:54:31 on dist-test-slave-c51v"
server_key: "dfa55e58ba544d57fbde772c3046aba0"
server_key_iv: "6d06a957397b5c1a771a5576e2e107f7"
server_key_version: "encryptionkey@0"
I20260929 15:54:31.901279 30251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "5f7986e698c643f69a2ab6aa83873ae0"
format_stamp: "Formatted at 2026-09-29 15:54:31 on dist-test-slave-c51v"
server_key: "dfa55e58ba544d57fbde772c3046aba0"
server_key_iv: "6d06a957397b5c1a771a5576e2e107f7"
server_key_version: "encryptionkey@0"
I20260929 15:54:31.906905 30251 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 15:54:31.910662 30266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:31.912186 30251 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:54:31.912379 30251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "5f7986e698c643f69a2ab6aa83873ae0"
format_stamp: "Formatted at 2026-09-29 15:54:31 on dist-test-slave-c51v"
server_key: "dfa55e58ba544d57fbde772c3046aba0"
server_key_iv: "6d06a957397b5c1a771a5576e2e107f7"
server_key_version: "encryptionkey@0"
I20260929 15:54:31.912590 30251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:31.932870 30251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:31.937348 30251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:31.937652 30251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:31.938625 30251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:31.940063 30251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:31.940135 30251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:31.940304 30251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:31.940353 30251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:31.971625 30251 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:32989
I20260929 15:54:31.971670 30378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:32989 every 8 connection(s)
I20260929 15:54:31.972923 30251 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:54:31.977099 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 30251
I20260929 15:54:31.977293 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:54:31.977865 16147 external_mini_cluster.cc:1428] Setting key f58f7472907e677dd1f45d061a6c818a
Sep 29 15:54:31 dist-test-slave-c51v krb5kdc[30011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697271, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:31.994421 30247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:31.975485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:41921 (local address 127.15.196.254:44615)
0929 15:54:31.975669 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:31.975673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:31.976888 (+ 1215us) server_negotiation.cc:407] Connection header received
0929 15:54:31.979246 (+ 2358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:31.979251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:31.979352 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:31.979447 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:31.981179 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:31.982378 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:31.983972 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:31.984785 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:31.984926 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:31.988480 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:31.988485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:31.988488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:31.988520 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:31.991049 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:31.991964 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:31.991968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:31.991971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:31.992097 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:31.992578 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:31.992581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:31.992583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:31.992924 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:31.993078 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:31.993832 (+ 754us) server_negotiation.cc:299] Negotiation successful
0929 15:54:31.994064 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 15:54:31.995606 30381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:31.975985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:41921)
0929 15:54:31.976697 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:31.976747 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:31.978990 (+ 2243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:31.979698 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:31.979712 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:31.980443 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:31.980998 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:31.981014 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:31.982512 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:31.982516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:31.983833 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:31.983845 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:31.984983 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:31.984987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:31.985179 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:31.985899 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:31.985924 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:31.988247 (+ 2323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:31.991220 (+ 2973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:31.991229 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:31.991247 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:31.991832 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:31.992225 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:31.992230 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:31.992236 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:31.992428 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:31.993061 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:31.993068 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:31.993463 (+ 395us) client_negotiation.cc:769] Sending connection context
0929 15:54:31.993732 (+ 269us) client_negotiation.cc:240] Negotiation successful
0929 15:54:31.994358 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":259,"threads_started":1}
I20260929 15:54:31.997166 30379 heartbeater.cc:347] Connected to a master server at 127.15.196.254:44615
I20260929 15:54:31.997555 30379 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:31.998306 30379 heartbeater.cc:511] Master 127.15.196.254:44615 requested a full tablet report, sending...
I20260929 15:54:31.999933 30056 ts_manager.cc:194] Registered new tserver with Master: 5f7986e698c643f69a2ab6aa83873ae0 (127.15.196.194:32989)
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
I20260929 15:54:32.001188 30056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:41921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:54:32.030457 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44615
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:35523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:54:32.356617 30384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:32.356909 30384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:32.357049 30384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:32.364341 30384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:32.364486 30384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:32.364669 30384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:54:32.374400 30384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:35523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:44615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:32.375787 30384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:32.377138 30384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:32.390350 30389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:32.390391 30390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:32.390321 30392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:32.391057 30384 server_base.cc:1034] running on GCE node
I20260929 15:54:32.391801 30384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:32.392879 30384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:32.394091 30384 hybrid_clock.cc:648] HybridClock initialized: now 1790697272394038 us; error 45 us; skew 500 ppm
I20260929 15:54:32.394361 30384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:32 dist-test-slave-c51v krb5kdc[30011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697272, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:32.397509 30384 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:54:32.398944 30384 webserver.cc:460] Webserver started at http://127.15.196.195:44869/ using document root <none> and password file <none>
I20260929 15:54:32.399722 30384 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:32.399924 30384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:32.400249 30384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:54:32.403367 30384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "ac90d00b24f44f02a5c825ad982e17e9"
format_stamp: "Formatted at 2026-09-29 15:54:32 on dist-test-slave-c51v"
server_key: "401debde1357af855a48da25a1b4f2cc"
server_key_iv: "69761b234e707668721f1d8ee68101b7"
server_key_version: "encryptionkey@0"
I20260929 15:54:32.404186 30384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "ac90d00b24f44f02a5c825ad982e17e9"
format_stamp: "Formatted at 2026-09-29 15:54:32 on dist-test-slave-c51v"
server_key: "401debde1357af855a48da25a1b4f2cc"
server_key_iv: "69761b234e707668721f1d8ee68101b7"
server_key_version: "encryptionkey@0"
I20260929 15:54:32.409284 30384 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260929 15:54:32.412763 30399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:32.414069 30384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 15:54:32.414261 30384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "ac90d00b24f44f02a5c825ad982e17e9"
format_stamp: "Formatted at 2026-09-29 15:54:32 on dist-test-slave-c51v"
server_key: "401debde1357af855a48da25a1b4f2cc"
server_key_iv: "69761b234e707668721f1d8ee68101b7"
server_key_version: "encryptionkey@0"
I20260929 15:54:32.414446 30384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:32.433609 30384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:32.437465 30384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:32.437724 30384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:32.438611 30384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:32.439881 30384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:32.439952 30384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:32.440034 30384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:32.440076 30384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:32.473735 30384 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:44415
I20260929 15:54:32.473775 30511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:44415 every 8 connection(s)
I20260929 15:54:32.475011 30384 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:54:32.483254 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 30384
I20260929 15:54:32.483500 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:54:32.483974 16147 external_mini_cluster.cc:1428] Setting key 6a37c1f4397d85af7062f00f8b9ed8e6
Sep 29 15:54:32 dist-test-slave-c51v krb5kdc[30011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697272, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:32.489451 30245 heartbeater.cc:503] Master 127.15.196.254:44615 was elected leader, sending a full tablet report...
I20260929 15:54:32.495790 30247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:32.477620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:35991 (local address 127.15.196.254:44615)
0929 15:54:32.477798 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:32.477803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:32.478914 (+ 1111us) server_negotiation.cc:407] Connection header received
0929 15:54:32.480856 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:32.480863 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:32.480950 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:32.481069 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:32.482573 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.483508 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.485329 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.485994 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.486144 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:32.489939 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:32.489947 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:32.489951 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:32.489993 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:32.492984 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:32.493713 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:32.493716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:32.493718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:32.493839 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:32.494291 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:32.494294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:32.494296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:32.494554 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:32.494664 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:32.495363 (+ 699us) server_negotiation.cc:299] Negotiation successful
0929 15:54:32.495541 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 15:54:32.496795 30514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:32.478054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:35991)
0929 15:54:32.478757 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:32.478799 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:32.480550 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:32.481303 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:32.481312 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:32.481815 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:32.482401 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:32.482419 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.483715 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.483719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:32.485171 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:32.485183 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.486261 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.486269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:32.486557 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:32.487353 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:32.487381 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:32.489717 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:32.493117 (+ 3400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:32.493124 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:32.493138 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:32.493615 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:32.493972 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:32.493979 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:32.493985 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:32.494198 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:32.494658 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:32.494664 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:32.494978 (+ 314us) client_negotiation.cc:769] Sending connection context
0929 15:54:32.495217 (+ 239us) client_negotiation.cc:240] Negotiation successful
0929 15:54:32.495673 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":275,"threads_started":1}
I20260929 15:54:32.498298 30512 heartbeater.cc:347] Connected to a master server at 127.15.196.254:44615
I20260929 15:54:32.498661 30512 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:32.499575 30512 heartbeater.cc:511] Master 127.15.196.254:44615 requested a full tablet report, sending...
I20260929 15:54:32.501163 30056 ts_manager.cc:194] Registered new tserver with Master: ac90d00b24f44f02a5c825ad982e17e9 (127.15.196.195:44415)
I20260929 15:54:32.501931 30056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:35991
I20260929 15:54:32.512455 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:54:32.530230 30247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:32.517111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44198 (local address 127.15.196.254:44615)
0929 15:54:32.517321 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:32.517328 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:32.517777 (+ 449us) server_negotiation.cc:407] Connection header received
0929 15:54:32.517997 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:32.518004 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:32.518078 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:32.518199 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:32.519830 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.520569 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.521878 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.522319 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.522459 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:32.524956 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:32.524961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:32.524964 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:32.524997 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:32.527464 (+ 2467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:32.528139 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:32.528145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:32.528149 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:32.528283 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:32.528685 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:32.528690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:32.528693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:32.529062 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:32.529333 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:32.529683 (+ 350us) server_negotiation.cc:299] Negotiation successful
0929 15:54:32.529908 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 15:54:32.534232 30056 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:54:32.541134 30056 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:54:32.568261 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:32.554221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49846 (local address 127.15.196.193:46253)
0929 15:54:32.554851 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:32.554860 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:32.555402 (+ 542us) server_negotiation.cc:407] Connection header received
0929 15:54:32.555683 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:32.555695 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:32.556015 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:32.556212 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:32.557475 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.559659 (+ 2184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.564003 (+ 4344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.566092 (+ 2089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.566307 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:32.566572 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:32.567488 (+ 916us) server_negotiation.cc:299] Negotiation successful
0929 15:54:32.567755 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":333,"threads_started":1}
I20260929 15:54:32.569504 30527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:32.553613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:46253 (local address 127.0.0.1:49846)
0929 15:54:32.555264 (+ 1651us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:32.555312 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:32.555429 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:32.556622 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:32.556633 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:32.556676 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:32.557222 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:32.557239 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.559903 (+ 2664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.559912 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:32.563765 (+ 3853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:32.563787 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.566800 (+ 3013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.566808 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:32.567099 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:32.567170 (+ 71us) client_negotiation.cc:769] Sending connection context
0929 15:54:32.568837 (+ 1667us) client_negotiation.cc:240] Negotiation successful
0929 15:54:32.569045 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1534,"mutex_wait_us":1,"thread_start_us":1421,"threads_started":1}
I20260929 15:54:32.570041 30524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:32.553009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:32989 (local address 127.0.0.1:51530)
0929 15:54:32.554990 (+ 1981us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:32.555060 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:32.555388 (+ 328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:32.557871 (+ 2483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:32.557879 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:32.557913 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:32.558458 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:32.558473 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.562468 (+ 3995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.562475 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:32.566805 (+ 4330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:32.566825 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.569230 (+ 2405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.569238 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:32.569418 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:32.569460 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 15:54:32.569562 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 15:54:32.569708 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1664,"thread_start_us":1535,"threads_started":1}
I20260929 15:54:32.571157 30525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:32.553222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51530 (local address 127.15.196.194:32989)
0929 15:54:32.553803 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:32.553814 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:32.555182 (+ 1368us) server_negotiation.cc:407] Connection header received
0929 15:54:32.555678 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:32.555690 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:32.555963 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:32.557487 (+ 1524us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:32.558712 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.562227 (+ 3515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.567037 (+ 4810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.568755 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.568902 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:32.569082 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:32.570693 (+ 1611us) server_negotiation.cc:299] Negotiation successful
0929 15:54:32.570873 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":743,"threads_started":1}
I20260929 15:54:32.573740 30180 tablet_service.cc:1467] Processing CreateTablet for tablet 112ffaa86c2a4ca7a96990275ab67ead (DEFAULT_TABLE table=test-table [id=549ce077907946a496a67c7bb88e1142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:54:32.574640 30314 tablet_service.cc:1467] Processing CreateTablet for tablet 112ffaa86c2a4ca7a96990275ab67ead (DEFAULT_TABLE table=test-table [id=549ce077907946a496a67c7bb88e1142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:54:32.575192 30180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112ffaa86c2a4ca7a96990275ab67ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:32.575990 30314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112ffaa86c2a4ca7a96990275ab67ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:32.577163 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:32.555450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:44415 (local address 127.0.0.1:44414)
0929 15:54:32.568830 (+ 13380us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:32.568855 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:32.568973 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:32.569959 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:32.569963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:32.569987 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:32.570407 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:32.570420 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.572297 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.572303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:32.574281 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:32.574296 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.576414 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.576421 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:32.576591 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:32.576623 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 15:54:32.576714 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 15:54:32.576849 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13228,"thread_start_us":13421,"threads_started":1}
I20260929 15:54:32.577630 30529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:32.555866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.15.196.195:44415)
0929 15:54:32.568170 (+ 12304us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:32.568180 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:32.569121 (+ 941us) server_negotiation.cc:407] Connection header received
0929 15:54:32.569236 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:32.569242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:32.569454 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:32.569669 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:32.570616 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.572144 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.574454 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.575892 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.576076 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:32.576235 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:32.576848 (+ 613us) server_negotiation.cc:299] Negotiation successful
0929 15:54:32.577011 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12089,"thread_start_us":12313,"threads_started":1}
I20260929 15:54:32.581355 30447 tablet_service.cc:1467] Processing CreateTablet for tablet 112ffaa86c2a4ca7a96990275ab67ead (DEFAULT_TABLE table=test-table [id=549ce077907946a496a67c7bb88e1142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:54:32.582785 30447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112ffaa86c2a4ca7a96990275ab67ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:32.588004 30530 tablet_bootstrap.cc:492] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0: Bootstrap starting.
I20260929 15:54:32.589802 30531 tablet_bootstrap.cc:492] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7: Bootstrap starting.
I20260929 15:54:32.591939 30530 tablet_bootstrap.cc:654] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:32.593272 30531 tablet_bootstrap.cc:654] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:32.593443 30530 log.cc:824] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:32.594496 30531 log.cc:824] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:32.596663 30530 tablet_bootstrap.cc:492] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0: No bootstrap required, opened a new log
I20260929 15:54:32.596784 30534 tablet_bootstrap.cc:492] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9: Bootstrap starting.
I20260929 15:54:32.597069 30530 ts_tablet_manager.cc:1397] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260929 15:54:32.597486 30531 tablet_bootstrap.cc:492] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7: No bootstrap required, opened a new log
I20260929 15:54:32.597739 30531 ts_tablet_manager.cc:1397] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 15:54:32.600065 30534 tablet_bootstrap.cc:654] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:32.601195 30534 log.cc:824] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:32.604593 30534 tablet_bootstrap.cc:492] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9: No bootstrap required, opened a new log
I20260929 15:54:32.604877 30534 ts_tablet_manager.cc:1397] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260929 15:54:32.607963 30530 raft_consensus.cc:348] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:32.608466 30530 raft_consensus.cc:374] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:32.608577 30530 raft_consensus.cc:729] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f7986e698c643f69a2ab6aa83873ae0, State: Initialized, Role: FOLLOWER
I20260929 15:54:32.608376 30531 raft_consensus.cc:348] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:32.608801 30531 raft_consensus.cc:374] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:32.608896 30531 raft_consensus.cc:729] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d24a657d04014827a741baae1dcd09a7, State: Initialized, Role: FOLLOWER
I20260929 15:54:32.609249 30530 consensus_queue.cc:260] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:32.609436 30531 consensus_queue.cc:260] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:32.610447 30379 heartbeater.cc:503] Master 127.15.196.254:44615 was elected leader, sending a full tablet report...
I20260929 15:54:32.611470 30530 ts_tablet_manager.cc:1428] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260929 15:54:32.613121 30531 ts_tablet_manager.cc:1428] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260929 15:54:32.615005 30534 raft_consensus.cc:348] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:32.615520 30534 raft_consensus.cc:374] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:32.615648 30534 raft_consensus.cc:729] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac90d00b24f44f02a5c825ad982e17e9, State: Initialized, Role: FOLLOWER
I20260929 15:54:32.616291 30534 consensus_queue.cc:260] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:32.618350 30512 heartbeater.cc:503] Master 127.15.196.254:44615 was elected leader, sending a full tablet report...
I20260929 15:54:32.619150 30534 ts_tablet_manager.cc:1428] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9: Time spent starting tablet: real 0.014s user 0.008s sys 0.005s
W20260929 15:54:32.710680 30246 tablet.cc:2369] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 15:54:32.725015 30380 tablet.cc:2369] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 15:54:32.726588 30513 tablet.cc:2369] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:54:32.758291 30537 raft_consensus.cc:482] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:54:32.758632 30537 raft_consensus.cc:504] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:32.760422 30537 leader_election.cc:288] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f7986e698c643f69a2ab6aa83873ae0 (127.15.196.194:32989), ac90d00b24f44f02a5c825ad982e17e9 (127.15.196.195:44415)
I20260929 15:54:32.767956 30525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:32.760970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:32873 (local address 127.15.196.194:32989)
0929 15:54:32.761159 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:32.761164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:32.761664 (+ 500us) server_negotiation.cc:407] Connection header received
0929 15:54:32.761927 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:32.761947 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:32.762058 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:32.762125 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:32.762949 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.764007 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.766019 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.766917 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.767042 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:32.767102 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:32.767538 (+ 436us) server_negotiation.cc:299] Negotiation successful
0929 15:54:32.767687 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 15:54:32.768115 30539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:32.760854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:32989 (local address 127.15.196.193:32873)
0929 15:54:32.761553 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:32.761575 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:32.761718 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:32.762334 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:32.762337 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:32.762360 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:32.762776 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:32.762791 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.764178 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.764182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:32.765863 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:32.765875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.767132 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.767138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:32.767359 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:32.767397 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 15:54:32.767487 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 15:54:32.767591 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"mutex_wait_us":27,"spinlock_wait_cycles":44416,"thread_start_us":371,"threads_started":1}
I20260929 15:54:32.768155 30529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:32.762008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:58171 (local address 127.15.196.195:44415)
0929 15:54:32.762232 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:32.762239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:32.762441 (+ 202us) server_negotiation.cc:407] Connection header received
0929 15:54:32.762686 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:32.762697 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:32.762757 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:32.762850 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:32.763444 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.764517 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.766304 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.767137 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.767323 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:32.767401 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:32.767755 (+ 354us) server_negotiation.cc:299] Negotiation successful
0929 15:54:32.767890 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260929 15:54:32.768235 30540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:32.761712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:44415 (local address 127.15.196.193:58171)
0929 15:54:32.762329 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:32.762347 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:32.762526 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:32.762876 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:32.762880 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:32.762904 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:32.763228 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:32.763235 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.764652 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.764655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:32.766152 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:32.766166 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.767408 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.767413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:32.767582 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:32.767614 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 15:54:32.767705 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 15:54:32.767844 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":16,"thread_start_us":298,"threads_started":1}
I20260929 15:54:32.769353 30334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "112ffaa86c2a4ca7a96990275ab67ead" candidate_uuid: "d24a657d04014827a741baae1dcd09a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f7986e698c643f69a2ab6aa83873ae0" is_pre_election: true
I20260929 15:54:32.769397 30467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "112ffaa86c2a4ca7a96990275ab67ead" candidate_uuid: "d24a657d04014827a741baae1dcd09a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac90d00b24f44f02a5c825ad982e17e9" is_pre_election: true
I20260929 15:54:32.769869 30334 raft_consensus.cc:2454] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d24a657d04014827a741baae1dcd09a7 in term 0.
I20260929 15:54:32.769869 30467 raft_consensus.cc:2454] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d24a657d04014827a741baae1dcd09a7 in term 0.
I20260929 15:54:32.770507 30135 leader_election.cc:302] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f7986e698c643f69a2ab6aa83873ae0, d24a657d04014827a741baae1dcd09a7; no voters:
I20260929 15:54:32.770910 30537 raft_consensus.cc:2781] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:54:32.771056 30537 raft_consensus.cc:482] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:54:32.771188 30537 raft_consensus.cc:3037] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:32.773394 30537 raft_consensus.cc:504] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:32.773988 30537 leader_election.cc:288] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [CANDIDATE]: Term 1 election: Requested vote from peers 5f7986e698c643f69a2ab6aa83873ae0 (127.15.196.194:32989), ac90d00b24f44f02a5c825ad982e17e9 (127.15.196.195:44415)
I20260929 15:54:32.774581 30334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "112ffaa86c2a4ca7a96990275ab67ead" candidate_uuid: "d24a657d04014827a741baae1dcd09a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f7986e698c643f69a2ab6aa83873ae0"
I20260929 15:54:32.774582 30467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "112ffaa86c2a4ca7a96990275ab67ead" candidate_uuid: "d24a657d04014827a741baae1dcd09a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac90d00b24f44f02a5c825ad982e17e9"
I20260929 15:54:32.774787 30467 raft_consensus.cc:3037] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:32.774780 30334 raft_consensus.cc:3037] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:32.777021 30334 raft_consensus.cc:2454] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d24a657d04014827a741baae1dcd09a7 in term 1.
I20260929 15:54:32.777037 30467 raft_consensus.cc:2454] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d24a657d04014827a741baae1dcd09a7 in term 1.
I20260929 15:54:32.777436 30135 leader_election.cc:302] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f7986e698c643f69a2ab6aa83873ae0, d24a657d04014827a741baae1dcd09a7; no voters:
I20260929 15:54:32.777734 30537 raft_consensus.cc:2781] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:54:32.778194 30537 raft_consensus.cc:686] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [term 1 LEADER]: Becoming Leader. State: Replica: d24a657d04014827a741baae1dcd09a7, State: Running, Role: LEADER
I20260929 15:54:32.778689 30537 consensus_queue.cc:237] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:32.783550 30054 catalog_manager.cc:5495] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d24a657d04014827a741baae1dcd09a7 (127.15.196.193). New cstate: current_term: 1 leader_uuid: "d24a657d04014827a741baae1dcd09a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } health_report { overall_health: HEALTHY } } }
I20260929 15:54:32.802852 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:32.797259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49854 (local address 127.15.196.193:46253)
0929 15:54:32.797471 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:32.797477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:32.797495 (+ 18us) server_negotiation.cc:407] Connection header received
0929 15:54:32.797729 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:32.797733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:32.797818 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:32.797968 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:54:32.798366 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.799093 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.800642 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.801065 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.801203 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:32.801739 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:54:32.802255 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:54:32.802421 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:32.802472 (+ 51us) server_negotiation.cc:299] Negotiation successful
0929 15:54:32.802556 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 15:54:32.812476 30467 raft_consensus.cc:1261] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9 [term 1 FOLLOWER]: Refusing update from remote peer d24a657d04014827a741baae1dcd09a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:54:32.812813 30334 raft_consensus.cc:1261] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0 [term 1 FOLLOWER]: Refusing update from remote peer d24a657d04014827a741baae1dcd09a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:54:32.813428 30537 consensus_queue.cc:1035] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:54:32.813786 30541 consensus_queue.cc:1035] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:54:32.831521 30546 mvcc.cc:205] Tried to move back new op lower bound from 7334696029425590272 to 7334696029305327616. Current Snapshot: MvccSnapshot[applied={T|T < 7334696029425590272}]
I20260929 15:54:32.841682 30056 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:54:32.841984 30056 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:54:32.845817 30056 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e: Sending DeleteTablet for 3 replicas of tablet 112ffaa86c2a4ca7a96990275ab67ead
I20260929 15:54:32.847157 30314 tablet_service.cc:1514] Processing DeleteTablet for tablet 112ffaa86c2a4ca7a96990275ab67ead with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:54:32 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:51530
I20260929 15:54:32.847379 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 30117
I20260929 15:54:32.847551 30180 tablet_service.cc:1514] Processing DeleteTablet for tablet 112ffaa86c2a4ca7a96990275ab67ead with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:54:32 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:49846
I20260929 15:54:32.847908 30447 tablet_service.cc:1514] Processing DeleteTablet for tablet 112ffaa86c2a4ca7a96990275ab67ead with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:54:32 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:44414
I20260929 15:54:32.848848 30555 tablet_replica.cc:330] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9: stopping tablet replica
I20260929 15:54:32.849444 30555 raft_consensus.cc:2229] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:54:32.850157 30555 raft_consensus.cc:2258] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:54:32.852603 30555 ts_tablet_manager.cc:1905] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:54:32.859637 30553 tablet_replica.cc:330] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0: stopping tablet replica
I20260929 15:54:32.860126 30553 raft_consensus.cc:2229] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:54:32.860654 30553 raft_consensus.cc:2258] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:54:32.860879 30555 ts_tablet_manager.cc:1918] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:54:32.861099 30555 log.cc:1196] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/112ffaa86c2a4ca7a96990275ab67ead
I20260929 15:54:32.861769 30555 ts_tablet_manager.cc:1939] T 112ffaa86c2a4ca7a96990275ab67ead P ac90d00b24f44f02a5c825ad982e17e9: Deleting consensus metadata
I20260929 15:54:32.863862 30042 catalog_manager.cc:4841] TS ac90d00b24f44f02a5c825ad982e17e9 (127.15.196.195:44415): tablet 112ffaa86c2a4ca7a96990275ab67ead (table test-table [id=549ce077907946a496a67c7bb88e1142]) successfully deleted
I20260929 15:54:32.864614 30553 ts_tablet_manager.cc:1905] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:54:32.869282 30554 tablet_replica.cc:330] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7: stopping tablet replica
I20260929 15:54:32.869774 30554 raft_consensus.cc:2229] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:54:32.870563 30554 raft_consensus.cc:2258] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:54:32.872432 30553 ts_tablet_manager.cc:1918] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:54:32.872610 30553 log.cc:1196] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/112ffaa86c2a4ca7a96990275ab67ead
I20260929 15:54:32.873139 30554 ts_tablet_manager.cc:1905] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:54:32.873190 30553 ts_tablet_manager.cc:1939] T 112ffaa86c2a4ca7a96990275ab67ead P 5f7986e698c643f69a2ab6aa83873ae0: Deleting consensus metadata
Sep 29 15:54:32 dist-test-slave-c51v krb5kdc[30011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:54:32.875268 30042 catalog_manager.cc:4841] TS 5f7986e698c643f69a2ab6aa83873ae0 (127.15.196.194:32989): tablet 112ffaa86c2a4ca7a96990275ab67ead (table test-table [id=549ce077907946a496a67c7bb88e1142]) successfully deleted
I20260929 15:54:32.884974 30554 ts_tablet_manager.cc:1918] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:54:32.885234 30554 log.cc:1196] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/112ffaa86c2a4ca7a96990275ab67ead
I20260929 15:54:32.886111 30554 ts_tablet_manager.cc:1939] T 112ffaa86c2a4ca7a96990275ab67ead P d24a657d04014827a741baae1dcd09a7: Deleting consensus metadata
I20260929 15:54:32.886897 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:32.851660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49864 (local address 127.15.196.193:46253)
0929 15:54:32.851841 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:32.851850 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:32.857412 (+ 5562us) server_negotiation.cc:407] Connection header received
0929 15:54:32.857607 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:32.857612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:32.857737 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:32.857851 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:32.865781 (+ 7930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.867003 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.869264 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:32.870066 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:32.870194 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:32.876812 (+ 6618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:32.876828 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:32.876881 (+ 53us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:32.876988 (+ 107us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:32.882342 (+ 5354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:32.883382 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:32.883395 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:32.883399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:32.883626 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:32.884102 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:32.884112 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:32.884118 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:32.884657 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:32.884932 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:32.886246 (+ 1314us) server_negotiation.cc:299] Negotiation successful
0929 15:54:32.886483 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 15:54:32.887676 30240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:32.888025 30041 catalog_manager.cc:4841] TS d24a657d04014827a741baae1dcd09a7 (127.15.196.193:46253): tablet 112ffaa86c2a4ca7a96990275ab67ead (table test-table [id=549ce077907946a496a67c7bb88e1142]) successfully deleted
I20260929 15:54:32.989436 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 30117
I20260929 15:54:33.007764 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 30251
Sep 29 15:54:33 dist-test-slave-c51v krb5kdc[30011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:54:33.023793 30525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:33.008513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51532 (local address 127.15.196.194:32989)
0929 15:54:33.008705 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:33.008713 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:33.008734 (+ 21us) server_negotiation.cc:407] Connection header received
0929 15:54:33.008957 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:33.008962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:33.009043 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:33.009150 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:33.010489 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:33.011253 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:33.012576 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:33.012967 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:33.013101 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:33.018101 (+ 5000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:33.018140 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:33.018156 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:33.018192 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:33.020988 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:33.021685 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:33.021692 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:33.021694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:33.021821 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:33.022197 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:33.022204 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:33.022207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:33.022587 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:33.022887 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:33.023201 (+ 314us) server_negotiation.cc:299] Negotiation successful
0929 15:54:33.023439 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 15:54:33.024441 30374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:33.116312 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 30251
I20260929 15:54:33.133844 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 30384
Sep 29 15:54:33 dist-test-slave-c51v krb5kdc[30011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:54:33.148689 30529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:33.134657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44430 (local address 127.15.196.195:44415)
0929 15:54:33.134837 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:33.134841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:33.134859 (+ 18us) server_negotiation.cc:407] Connection header received
0929 15:54:33.134988 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:33.134992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:33.135054 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:33.135148 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:33.136491 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:33.137219 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:33.138453 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:33.138826 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:33.138913 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:33.143159 (+ 4246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:33.143175 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:33.143191 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:33.143227 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:33.145811 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:33.146517 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:33.146525 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:33.146528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:33.146680 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:33.147077 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:33.147082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:33.147087 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:33.147490 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:33.147800 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:33.148118 (+ 318us) server_negotiation.cc:299] Negotiation successful
0929 15:54:33.148339 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 15:54:33.149346 30507 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:33.239060 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 30384
I20260929 15:54:33.256176 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 30024
I20260929 15:54:33.257321 30086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:33.370424 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 30024
I20260929 15:54:33.388808 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:44615
--webserver_interface=127.15.196.254
--webserver_port=44833
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:35523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:44615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:54:33.722196 30561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:33.722559 30561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:33.722752 30561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:33.730039 30561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:54:33.730201 30561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:33.730319 30561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:33.730427 30561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:54:33.730496 30561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:54:33.740304 30561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:35523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:44615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:44615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=44833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:33.741676 30561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:33.743005 30561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:33.765007 30568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:33.764940 30567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:33.764976 30570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:33.765885 30561 server_base.cc:1034] running on GCE node
I20260929 15:54:33.766759 30561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:33.768416 30561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:33.769661 30561 hybrid_clock.cc:648] HybridClock initialized: now 1790697273769603 us; error 63 us; skew 500 ppm
I20260929 15:54:33.770064 30561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:33 dist-test-slave-c51v krb5kdc[30011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697273, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:33.773711 30561 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:54:33.775494 30561 webserver.cc:460] Webserver started at http://127.15.196.254:44833/ using document root <none> and password file <none>
I20260929 15:54:33.776284 30561 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:33.776453 30561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:33.783012 30561 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.007s sys 0.000s
I20260929 15:54:33.786096 30577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:33.787611 30561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:54:33.787803 30561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "4d9bd3eddcd34685a24d0466f4879f6e"
format_stamp: "Formatted at 2026-09-29 15:54:30 on dist-test-slave-c51v"
server_key: "4e27288f0b67470df12cc7f7d0a2177b"
server_key_iv: "b565c6203ccd626c48ac0b46be5c832e"
server_key_version: "encryptionkey@0"
I20260929 15:54:33.788902 30561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:33.832012 30561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:33.836846 30561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:33.837141 30561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:33.856837 30561 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:44615
I20260929 15:54:33.856844 30628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:44615 every 8 connection(s)
I20260929 15:54:33.858435 30561 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:54:33.862409 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 30561
I20260929 15:54:33.863205 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:46253
--local_ip_for_outbound_sockets=127.15.196.193
--tserver_master_addrs=127.15.196.254:44615
--webserver_port=38839
--webserver_interface=127.15.196.193
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:35523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:54:33.865646 30629 sys_catalog.cc:263] Verifying existing consensus state
I20260929 15:54:33.868980 30629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e: Bootstrap starting.
I20260929 15:54:33.889000 30629 log.cc:824] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:33.903859 30629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 15:54:33.904510 30629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e: Bootstrap complete.
I20260929 15:54:33.915552 30629 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } }
I20260929 15:54:33.916610 30629 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9bd3eddcd34685a24d0466f4879f6e, State: Initialized, Role: FOLLOWER
I20260929 15:54:33.917344 30629 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } }
I20260929 15:54:33.917676 30629 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:54:33.917837 30629 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:54:33.918046 30629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 1 FOLLOWER]: Advancing to term 2
I20260929 15:54:33.922608 30629 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } }
I20260929 15:54:33.923220 30629 leader_election.cc:302] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d9bd3eddcd34685a24d0466f4879f6e; no voters:
I20260929 15:54:33.924015 30629 leader_election.cc:288] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 15:54:33.924185 30633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 2 FOLLOWER]: Leader election won for term 2
I20260929 15:54:33.925433 30633 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 2 LEADER]: Becoming Leader. State: Replica: 4d9bd3eddcd34685a24d0466f4879f6e, State: Running, Role: LEADER
I20260929 15:54:33.926173 30633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } }
I20260929 15:54:33.926671 30629 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:54:33.929065 30634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } } }
I20260929 15:54:33.929601 30634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:33.933651 30635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d9bd3eddcd34685a24d0466f4879f6e. Latest consensus state: current_term: 2 leader_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } } }
I20260929 15:54:33.934188 30635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:33.935894 30639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:54:33.948071 30639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:54:33.950546 30639 catalog_manager.cc:1259] Loaded cluster ID: 8dfd4078f4b64f8cb4a2236da0e388ff
I20260929 15:54:33.950804 30639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:54:33.956076 30639 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260929 15:54:33.986809 30639 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260929 15:54:34.008752 30639 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:54:34.010785 30639 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e: Loaded TSK: 0
I20260929 15:54:34.012558 30639 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 15:54:34.223994 30631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:34.224334 30631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:34.224498 30631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:34.231868 30631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:34.232016 30631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:34.232191 30631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:54:34.242439 30631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:35523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:46253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=38839
--tserver_master_addrs=127.15.196.254:44615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:34.243835 30631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:34.245237 30631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:34.260072 30656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:34.260094 30657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:34.260828 30659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:34.264989 30631 server_base.cc:1034] running on GCE node
I20260929 15:54:34.265617 30631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:34.266651 30631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:34.267890 30631 hybrid_clock.cc:648] HybridClock initialized: now 1790697274267827 us; error 58 us; skew 500 ppm
I20260929 15:54:34.268208 30631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:34 dist-test-slave-c51v krb5kdc[30011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697274, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:34.271373 30631 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:54:34.272982 30631 webserver.cc:460] Webserver started at http://127.15.196.193:38839/ using document root <none> and password file <none>
I20260929 15:54:34.273693 30631 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:34.273833 30631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:34.279371 30631 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:54:34.282240 30666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:34.283845 30631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:54:34.284020 30631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "d24a657d04014827a741baae1dcd09a7"
format_stamp: "Formatted at 2026-09-29 15:54:31 on dist-test-slave-c51v"
server_key: "acd541a656e3765dbe63472ebc5629d4"
server_key_iv: "5489a1599fcc40057dbd102d43bfbe4b"
server_key_version: "encryptionkey@0"
I20260929 15:54:34.285039 30631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:34.299885 30631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:34.304328 30631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:34.304623 30631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:34.305761 30631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:34.307238 30631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:34.307377 30631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260929 15:54:34.307508 30631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:34.307588 30631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:34.346119 30631 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:46253
I20260929 15:54:34.346165 30778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:46253 every 8 connection(s)
I20260929 15:54:34.347683 30631 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:54:34.356210 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 30631
I20260929 15:54:34.356829 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:32989
--local_ip_for_outbound_sockets=127.15.196.194
--tserver_master_addrs=127.15.196.254:44615
--webserver_port=43785
--webserver_interface=127.15.196.194
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:35523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 15:54:34 dist-test-slave-c51v krb5kdc[30011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697274, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:34.373771 30782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:34.350597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:47459)
0929 15:54:34.351479 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:34.351524 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:34.353178 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:34.354484 (+ 1306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:34.354491 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:34.354994 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:34.355573 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:34.355591 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:34.357240 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:34.357248 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:34.359492 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:34.359511 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:34.360756 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:34.360766 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:34.361106 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:34.361852 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:34.361886 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:34.364658 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:34.369438 (+ 4780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:34.369445 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:34.369456 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:34.369889 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:34.370471 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:34.370475 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:34.370478 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:34.370711 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:34.371757 (+ 1046us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:34.371763 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:34.372098 (+ 335us) client_negotiation.cc:769] Sending connection context
0929 15:54:34.372331 (+ 233us) client_negotiation.cc:240] Negotiation successful
0929 15:54:34.372748 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":332,"threads_started":1}
I20260929 15:54:34.374341 30781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:34.350510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:47459 (local address 127.15.196.254:44615)
0929 15:54:34.351232 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:34.351247 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:34.351658 (+ 411us) server_negotiation.cc:407] Connection header received
0929 15:54:34.353693 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:34.353705 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:34.354034 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:34.354312 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:34.355760 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:34.357095 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:34.359659 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:34.360517 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:34.360666 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:34.364970 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:34.364984 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:34.365005 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:34.365086 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:34.369272 (+ 4186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:34.370054 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:34.370066 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:34.370076 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:34.370285 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:34.370845 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:34.370852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:34.370858 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:34.371613 (+ 755us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:34.371825 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:34.372646 (+ 821us) server_negotiation.cc:299] Negotiation successful
0929 15:54:34.373129 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":276,"threads_started":1}
I20260929 15:54:34.377518 30779 heartbeater.cc:347] Connected to a master server at 127.15.196.254:44615
I20260929 15:54:34.377847 30779 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:34.378609 30779 heartbeater.cc:511] Master 127.15.196.254:44615 requested a full tablet report, sending...
I20260929 15:54:34.381135 30594 ts_manager.cc:194] Registered new tserver with Master: d24a657d04014827a741baae1dcd09a7 (127.15.196.193:46253)
I20260929 15:54:34.383594 30594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:47459
W20260929 15:54:34.688735 30783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:34.689070 30783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:34.689190 30783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:34.696806 30783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:34.696971 30783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:34.697156 30783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:54:34.706753 30783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:35523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:32989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=43785
--tserver_master_addrs=127.15.196.254:44615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:34.708132 30783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:34.709487 30783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:34.723098 30788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:34.723157 30791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:34.723179 30789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:34.724223 30783 server_base.cc:1034] running on GCE node
I20260929 15:54:34.724864 30783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:34.725814 30783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:34.727085 30783 hybrid_clock.cc:648] HybridClock initialized: now 1790697274727017 us; error 68 us; skew 500 ppm
I20260929 15:54:34.727553 30783 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:34 dist-test-slave-c51v krb5kdc[30011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697274, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:34.730850 30783 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:54:34.732409 30783 webserver.cc:460] Webserver started at http://127.15.196.194:43785/ using document root <none> and password file <none>
I20260929 15:54:34.733058 30783 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:34.733181 30783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:34.739066 30783 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 15:54:34.741734 30798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:34.743069 30783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 15:54:34.743206 30783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "5f7986e698c643f69a2ab6aa83873ae0"
format_stamp: "Formatted at 2026-09-29 15:54:31 on dist-test-slave-c51v"
server_key: "dfa55e58ba544d57fbde772c3046aba0"
server_key_iv: "6d06a957397b5c1a771a5576e2e107f7"
server_key_version: "encryptionkey@0"
I20260929 15:54:34.744194 30783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:34.774194 30783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:34.778738 30783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:34.779027 30783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:34.780197 30783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:34.781663 30783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:34.781809 30783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:34.781934 30783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:34.782017 30783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:34.832069 30783 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:32989
I20260929 15:54:34.832121 30910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:32989 every 8 connection(s)
I20260929 15:54:34.833526 30783 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:54:34.834015 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 30783
I20260929 15:54:34.834861 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:44415
--local_ip_for_outbound_sockets=127.15.196.195
--tserver_master_addrs=127.15.196.254:44615
--webserver_port=44869
--webserver_interface=127.15.196.195
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:35523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 15:54:34 dist-test-slave-c51v krb5kdc[30011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697274, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:34.866050 30913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:34.837784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:44447)
0929 15:54:34.839953 (+ 2169us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:34.840009 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:34.843060 (+ 3051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:34.844014 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:34.844030 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:34.844913 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:34.845850 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:34.845879 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:34.847461 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:34.847469 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:34.849963 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:34.849995 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:34.851210 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:34.851220 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:34.851611 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:34.852505 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:34.852541 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:34.856120 (+ 3579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:34.861095 (+ 4975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:34.861108 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:34.861128 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:34.861750 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:34.862287 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:34.862292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:34.862296 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:34.862539 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:34.863319 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:34.863327 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:34.863768 (+ 441us) client_negotiation.cc:769] Sending connection context
0929 15:54:34.864093 (+ 325us) client_negotiation.cc:240] Negotiation successful
0929 15:54:34.864659 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1803,"thread_start_us":1561,"threads_started":1}
I20260929 15:54:34.867389 30781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:34.837115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:44447 (local address 127.15.196.254:44615)
0929 15:54:34.837318 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:34.837322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:34.840152 (+ 2830us) server_negotiation.cc:407] Connection header received
0929 15:54:34.843441 (+ 3289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:34.843448 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:34.843566 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:34.843720 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:34.846067 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:34.847254 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:34.850166 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:34.850948 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:34.851094 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:34.856431 (+ 5337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:34.856440 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:34.856448 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:34.856503 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:34.860909 (+ 4406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:34.861907 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:34.861916 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:34.861921 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:34.862150 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:34.862660 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:34.862667 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:34.862673 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:34.863133 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:34.863361 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:34.866796 (+ 3435us) server_negotiation.cc:299] Negotiation successful
0929 15:54:34.867034 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260929 15:54:34.869009 30911 heartbeater.cc:347] Connected to a master server at 127.15.196.254:44615
I20260929 15:54:34.869500 30911 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:34.870590 30911 heartbeater.cc:511] Master 127.15.196.254:44615 requested a full tablet report, sending...
I20260929 15:54:34.872402 30594 ts_manager.cc:194] Registered new tserver with Master: 5f7986e698c643f69a2ab6aa83873ae0 (127.15.196.194:32989)
I20260929 15:54:34.873559 30594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:44447
W20260929 15:54:35.174538 30914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:35.174809 30914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:35.174906 30914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:35.182101 30914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:35.182221 30914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:35.182332 30914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:54:35.192018 30914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:35523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:44415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=44869
--tserver_master_addrs=127.15.196.254:44615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:35.193373 30914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:35.194654 30914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:35.207996 30920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:35.207955 30922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:35.208004 30919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:35.208989 30914 server_base.cc:1034] running on GCE node
I20260929 15:54:35.209589 30914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:35.210614 30914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:35.211833 30914 hybrid_clock.cc:648] HybridClock initialized: now 1790697275211796 us; error 52 us; skew 500 ppm
I20260929 15:54:35.212152 30914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:35 dist-test-slave-c51v krb5kdc[30011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697275, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:35.215201 30914 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:54:35.216694 30914 webserver.cc:460] Webserver started at http://127.15.196.195:44869/ using document root <none> and password file <none>
I20260929 15:54:35.217327 30914 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:35.217437 30914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:35.222795 30914 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 15:54:35.225449 30929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:35.226868 30914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:54:35.227042 30914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "ac90d00b24f44f02a5c825ad982e17e9"
format_stamp: "Formatted at 2026-09-29 15:54:32 on dist-test-slave-c51v"
server_key: "401debde1357af855a48da25a1b4f2cc"
server_key_iv: "69761b234e707668721f1d8ee68101b7"
server_key_version: "encryptionkey@0"
I20260929 15:54:35.228184 30914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:35.239593 30914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:35.243925 30914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:35.244148 30914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:35.245007 30914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:35.246202 30914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:35.246284 30914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:35.246361 30914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:35.246414 30914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:35.279533 30914 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:44415
I20260929 15:54:35.279632 31041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:44415 every 8 connection(s)
I20260929 15:54:35.281055 30914 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:54:35.281412 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 30914
Sep 29 15:54:35 dist-test-slave-c51v krb5kdc[30011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697275, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:35.299835 30781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.282439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44212 (local address 127.15.196.254:44615)
0929 15:54:35.282702 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:35.282712 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:35.283952 (+ 1240us) server_negotiation.cc:407] Connection header received
0929 15:54:35.284191 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:35.284197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:35.284282 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:35.284404 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:35.286937 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.288157 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.289676 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.290341 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.290486 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.293921 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:35.293927 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:35.293931 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:35.293964 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:35.296919 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:35.297593 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:35.297598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:35.297602 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:35.297805 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:35.298297 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:35.298301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:35.298303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:35.298594 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:35.298859 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:35.299199 (+ 340us) server_negotiation.cc:299] Negotiation successful
0929 15:54:35.299487 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":76}
I20260929 15:54:35.304139 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.284125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:40677 (local address 127.15.196.254:44615)
0929 15:54:35.287243 (+ 3118us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:35.287247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:35.287262 (+ 15us) server_negotiation.cc:407] Connection header received
0929 15:54:35.287922 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:35.287926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:35.287981 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:35.288075 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:35.289920 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.290745 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.293162 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.293690 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.293784 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.297233 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:35.297238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:35.297241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:35.297269 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:35.300422 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:35.302049 (+ 1627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:35.302054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:35.302057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:35.302194 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:35.302670 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:35.302673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:35.302675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:35.302915 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:35.303018 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:35.303722 (+ 704us) server_negotiation.cc:299] Negotiation successful
0929 15:54:35.303868 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":3021,"thread_start_us":2865,"threads_started":1}
I20260929 15:54:35.305061 31046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.284827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:40677)
0929 15:54:35.285632 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:35.285678 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:35.287654 (+ 1976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:35.288362 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:35.288370 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:35.288999 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:35.289740 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:35.289767 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.290899 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.290905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:35.293018 (+ 2113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:35.293036 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.293913 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.293919 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:35.294127 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.294709 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:35.294729 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:35.297014 (+ 2285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:35.300611 (+ 3597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:35.300620 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:35.300640 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:35.301236 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:35.302316 (+ 1080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:35.302322 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:35.302325 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:35.302562 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:35.303022 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:35.303027 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:35.303395 (+ 368us) client_negotiation.cc:769] Sending connection context
0929 15:54:35.303632 (+ 237us) client_negotiation.cc:240] Negotiation successful
0929 15:54:35.304000 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":340,"threads_started":1}
I20260929 15:54:35.306409 31042 heartbeater.cc:347] Connected to a master server at 127.15.196.254:44615
I20260929 15:54:35.306667 31042 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:35.307521 31042 heartbeater.cc:511] Master 127.15.196.254:44615 requested a full tablet report, sending...
I20260929 15:54:35.308857 30594 ts_manager.cc:194] Registered new tserver with Master: ac90d00b24f44f02a5c825ad982e17e9 (127.15.196.195:44415)
I20260929 15:54:35.309635 30594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:40677
I20260929 15:54:35.314412 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:54:35.320969 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.315400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44228 (local address 127.15.196.254:44615)
0929 15:54:35.315610 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:35.315619 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:35.315933 (+ 314us) server_negotiation.cc:407] Connection header received
0929 15:54:35.316105 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:35.316109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:35.316231 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:35.316349 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:54:35.316723 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.317410 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.318800 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.319202 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.319363 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.319805 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:54:35.320388 (+ 583us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:54:35.320495 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:35.320595 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 15:54:35.320658 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 15:54:35.321893 30594 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44228:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:54:35.331275 30594 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:54:35.362932 31057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.347635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51536 (local address 127.15.196.194:32989)
0929 15:54:35.354724 (+ 7089us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:35.354731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:35.354762 (+ 31us) server_negotiation.cc:407] Connection header received
0929 15:54:35.354841 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:35.354847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:35.355027 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:35.355238 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:35.355948 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.357005 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.359805 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.361272 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.361455 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.361658 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:35.362377 (+ 719us) server_negotiation.cc:299] Negotiation successful
0929 15:54:35.362558 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6897,"thread_start_us":6741,"threads_started":1}
I20260929 15:54:35.362955 31054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.344925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:32989 (local address 127.0.0.1:51536)
0929 15:54:35.353771 (+ 8846us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:35.353789 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:35.353907 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:35.355245 (+ 1338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:35.355253 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:35.355312 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:35.355745 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:35.355761 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.357177 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.357182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:35.359650 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:35.359664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.361807 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.361816 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:35.362083 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.362152 (+ 69us) client_negotiation.cc:769] Sending connection context
0929 15:54:35.362292 (+ 140us) client_negotiation.cc:240] Negotiation successful
0929 15:54:35.362504 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8728,"mutex_wait_us":24,"thread_start_us":2992,"threads_started":1}
I20260929 15:54:35.366267 31056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.344008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:46253 (local address 127.0.0.1:49870)
0929 15:54:35.345396 (+ 1388us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:35.345412 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:35.345523 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:35.358892 (+ 13369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:35.358900 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:35.358938 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:35.359399 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:35.359412 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.361326 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.361333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:35.363562 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:35.363577 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.365398 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.365404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:35.365558 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.365607 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 15:54:35.365727 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 15:54:35.365871 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1281,"mutex_wait_us":3,"thread_start_us":9506,"threads_started":1}
I20260929 15:54:35.366597 31052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.343474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:44415 (local address 127.0.0.1:44436)
0929 15:54:35.344521 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:35.344575 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:35.344730 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:35.360068 (+ 15338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:35.360073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:35.360098 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:35.360467 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:35.360482 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.362066 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.362074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:35.364052 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:35.364064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.365817 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.365823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:35.365990 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.366023 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 15:54:35.366124 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 15:54:35.366267 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"mutex_wait_us":8,"thread_start_us":652,"threads_started":1}
I20260929 15:54:35.366818 31053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.343744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44436 (local address 127.15.196.195:44415)
0929 15:54:35.359536 (+ 15792us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:35.359555 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:35.359587 (+ 32us) server_negotiation.cc:407] Connection header received
0929 15:54:35.359679 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:35.359685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:35.359894 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:35.360067 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:35.360681 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.361931 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.364203 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.365508 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.365810 (+ 302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.365958 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:35.366171 (+ 213us) server_negotiation.cc:299] Negotiation successful
0929 15:54:35.366323 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15621,"thread_start_us":15446,"threads_started":1}
I20260929 15:54:35.367403 31055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.344682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49870 (local address 127.15.196.193:46253)
0929 15:54:35.358304 (+ 13622us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:35.358313 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:35.358340 (+ 27us) server_negotiation.cc:407] Connection header received
0929 15:54:35.358433 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:35.358441 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:35.358679 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:35.358859 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:35.359590 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.361153 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.364119 (+ 2966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.365176 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.365399 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.365563 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:35.366958 (+ 1395us) server_negotiation.cc:299] Negotiation successful
0929 15:54:35.367109 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13461,"thread_start_us":13323,"threads_started":1}
I20260929 15:54:35.367687 30846 tablet_service.cc:1467] Processing CreateTablet for tablet 86be4b087da44acba2c72d3c8e1991f7 (DEFAULT_TABLE table=test-table [id=704e653341134a3f8d38d8ba9b359cfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:54:35.368968 30846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86be4b087da44acba2c72d3c8e1991f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:35.369935 30977 tablet_service.cc:1467] Processing CreateTablet for tablet 86be4b087da44acba2c72d3c8e1991f7 (DEFAULT_TABLE table=test-table [id=704e653341134a3f8d38d8ba9b359cfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:54:35.370961 30977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86be4b087da44acba2c72d3c8e1991f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:35.374327 30714 tablet_service.cc:1467] Processing CreateTablet for tablet 86be4b087da44acba2c72d3c8e1991f7 (DEFAULT_TABLE table=test-table [id=704e653341134a3f8d38d8ba9b359cfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:54:35.375555 30714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86be4b087da44acba2c72d3c8e1991f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:35.381294 31058 tablet_bootstrap.cc:492] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0: Bootstrap starting.
I20260929 15:54:35.384864 31058 tablet_bootstrap.cc:654] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:35.385860 31058 log.cc:824] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:35.386921 30779 heartbeater.cc:503] Master 127.15.196.254:44615 was elected leader, sending a full tablet report...
I20260929 15:54:35.389325 31060 tablet_bootstrap.cc:492] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9: Bootstrap starting.
I20260929 15:54:35.389353 31058 tablet_bootstrap.cc:492] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0: No bootstrap required, opened a new log
I20260929 15:54:35.389660 31058 ts_tablet_manager.cc:1397] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 15:54:35.391664 31061 tablet_bootstrap.cc:492] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7: Bootstrap starting.
I20260929 15:54:35.393659 31060 tablet_bootstrap.cc:654] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:35.394532 31061 tablet_bootstrap.cc:654] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:35.394917 31060 log.cc:824] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:35.395669 31061 log.cc:824] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:35.397943 31060 tablet_bootstrap.cc:492] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9: No bootstrap required, opened a new log
I20260929 15:54:35.398244 31060 ts_tablet_manager.cc:1397] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260929 15:54:35.400380 31061 tablet_bootstrap.cc:492] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7: No bootstrap required, opened a new log
I20260929 15:54:35.400642 31061 ts_tablet_manager.cc:1397] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260929 15:54:35.400336 31058 raft_consensus.cc:348] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:35.400782 31058 raft_consensus.cc:374] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:35.400880 31058 raft_consensus.cc:729] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f7986e698c643f69a2ab6aa83873ae0, State: Initialized, Role: FOLLOWER
I20260929 15:54:35.402343 31058 consensus_queue.cc:260] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:35.403491 30911 heartbeater.cc:503] Master 127.15.196.254:44615 was elected leader, sending a full tablet report...
I20260929 15:54:35.404119 31058 ts_tablet_manager.cc:1428] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260929 15:54:35.408396 31060 raft_consensus.cc:348] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:35.408792 31060 raft_consensus.cc:374] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:35.408892 31060 raft_consensus.cc:729] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac90d00b24f44f02a5c825ad982e17e9, State: Initialized, Role: FOLLOWER
I20260929 15:54:35.409377 31061 raft_consensus.cc:348] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:35.409526 31060 consensus_queue.cc:260] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:35.409785 31061 raft_consensus.cc:374] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:35.409926 31061 raft_consensus.cc:729] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d24a657d04014827a741baae1dcd09a7, State: Initialized, Role: FOLLOWER
I20260929 15:54:35.410521 31061 consensus_queue.cc:260] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:35.410876 31042 heartbeater.cc:503] Master 127.15.196.254:44615 was elected leader, sending a full tablet report...
I20260929 15:54:35.411550 31060 ts_tablet_manager.cc:1428] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260929 15:54:35.412254 31061 ts_tablet_manager.cc:1428] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260929 15:54:35.417776 31065 raft_consensus.cc:482] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:54:35.417968 31065 raft_consensus.cc:504] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:35.419394 31065 leader_election.cc:288] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f7986e698c643f69a2ab6aa83873ae0 (127.15.196.194:32989), d24a657d04014827a741baae1dcd09a7 (127.15.196.193:46253)
I20260929 15:54:35.425980 31055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.419873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:44981 (local address 127.15.196.193:46253)
0929 15:54:35.420083 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:35.420089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:35.420784 (+ 695us) server_negotiation.cc:407] Connection header received
0929 15:54:35.420915 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:35.420920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:35.420985 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:35.421063 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:35.421563 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.422581 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.424326 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.425057 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.425215 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.425295 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:35.425596 (+ 301us) server_negotiation.cc:299] Negotiation successful
0929 15:54:35.425731 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 15:54:35.426024 31046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.419876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:32989 (local address 127.15.196.195:33829)
0929 15:54:35.420297 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:35.420320 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:35.420463 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:35.420786 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:35.420792 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:35.420812 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:35.421003 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:35.421010 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.422312 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.422316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:35.424119 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:35.424131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.425248 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.425254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:35.425494 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.425543 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 15:54:35.425636 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 15:54:35.425755 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":70}
I20260929 15:54:35.426024 31057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.419795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:33829 (local address 127.15.196.194:32989)
0929 15:54:35.420135 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:35.420143 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:35.420497 (+ 354us) server_negotiation.cc:407] Connection header received
0929 15:54:35.420603 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:35.420613 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:35.420668 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:35.420763 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:35.421168 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.422185 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.424276 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.425066 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.425246 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.425324 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:35.425676 (+ 352us) server_negotiation.cc:299] Negotiation successful
0929 15:54:35.425810 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260929 15:54:35.426097 31067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.419905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:46253 (local address 127.15.196.195:44981)
0929 15:54:35.420672 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:35.420684 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:35.420804 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:35.421103 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:35.421110 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:35.421135 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:35.421416 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:35.421423 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.422710 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.422715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:35.424177 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:35.424214 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.425249 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.425255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:35.425443 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.425466 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 15:54:35.425535 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 15:54:35.425660 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":54,"thread_start_us":386,"threads_started":1}
I20260929 15:54:35.427011 30734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86be4b087da44acba2c72d3c8e1991f7" candidate_uuid: "ac90d00b24f44f02a5c825ad982e17e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d24a657d04014827a741baae1dcd09a7" is_pre_election: true
I20260929 15:54:35.427062 30866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86be4b087da44acba2c72d3c8e1991f7" candidate_uuid: "ac90d00b24f44f02a5c825ad982e17e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f7986e698c643f69a2ab6aa83873ae0" is_pre_election: true
I20260929 15:54:35.427500 30734 raft_consensus.cc:2454] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac90d00b24f44f02a5c825ad982e17e9 in term 0.
I20260929 15:54:35.427520 30866 raft_consensus.cc:2454] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac90d00b24f44f02a5c825ad982e17e9 in term 0.
I20260929 15:54:35.428139 30931 leader_election.cc:302] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac90d00b24f44f02a5c825ad982e17e9, d24a657d04014827a741baae1dcd09a7; no voters:
I20260929 15:54:35.428494 31065 raft_consensus.cc:2781] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:54:35.428633 31065 raft_consensus.cc:482] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:54:35.428733 31065 raft_consensus.cc:3037] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:35.430878 31065 raft_consensus.cc:504] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:35.431529 31065 leader_election.cc:288] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [CANDIDATE]: Term 1 election: Requested vote from peers 5f7986e698c643f69a2ab6aa83873ae0 (127.15.196.194:32989), d24a657d04014827a741baae1dcd09a7 (127.15.196.193:46253)
I20260929 15:54:35.431910 30866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86be4b087da44acba2c72d3c8e1991f7" candidate_uuid: "ac90d00b24f44f02a5c825ad982e17e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f7986e698c643f69a2ab6aa83873ae0"
I20260929 15:54:35.431993 30734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86be4b087da44acba2c72d3c8e1991f7" candidate_uuid: "ac90d00b24f44f02a5c825ad982e17e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d24a657d04014827a741baae1dcd09a7"
I20260929 15:54:35.432123 30866 raft_consensus.cc:3037] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:35.432246 30734 raft_consensus.cc:3037] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:35.434407 30866 raft_consensus.cc:2454] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac90d00b24f44f02a5c825ad982e17e9 in term 1.
I20260929 15:54:35.434422 30734 raft_consensus.cc:2454] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac90d00b24f44f02a5c825ad982e17e9 in term 1.
I20260929 15:54:35.434775 30932 leader_election.cc:302] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f7986e698c643f69a2ab6aa83873ae0, ac90d00b24f44f02a5c825ad982e17e9; no voters:
I20260929 15:54:35.435060 31065 raft_consensus.cc:2781] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:54:35.435525 31065 raft_consensus.cc:686] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [term 1 LEADER]: Becoming Leader. State: Replica: ac90d00b24f44f02a5c825ad982e17e9, State: Running, Role: LEADER
I20260929 15:54:35.435990 31065 consensus_queue.cc:237] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } } peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } }
I20260929 15:54:35.440696 30594 catalog_manager.cc:5495] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac90d00b24f44f02a5c825ad982e17e9 (127.15.196.195). New cstate: current_term: 1 leader_uuid: "ac90d00b24f44f02a5c825ad982e17e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac90d00b24f44f02a5c825ad982e17e9" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 } health_report { overall_health: UNKNOWN } } }
I20260929 15:54:35.470631 31053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.465626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44444 (local address 127.15.196.195:44415)
0929 15:54:35.465807 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:35.465811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:35.465886 (+ 75us) server_negotiation.cc:407] Connection header received
0929 15:54:35.466062 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:35.466065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:35.466152 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:35.466276 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:54:35.466610 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.467421 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.468783 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.469157 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.469246 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.469679 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:54:35.470176 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:54:35.470280 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:35.470356 (+ 76us) server_negotiation.cc:299] Negotiation successful
0929 15:54:35.470418 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 15:54:35.478523 30734 raft_consensus.cc:1261] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7 [term 1 FOLLOWER]: Refusing update from remote peer ac90d00b24f44f02a5c825ad982e17e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:54:35.478572 30866 raft_consensus.cc:1261] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0 [term 1 FOLLOWER]: Refusing update from remote peer ac90d00b24f44f02a5c825ad982e17e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:54:35.479534 31068 consensus_queue.cc:1035] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f7986e698c643f69a2ab6aa83873ae0" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 32989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:54:35.479897 31065 consensus_queue.cc:1035] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d24a657d04014827a741baae1dcd09a7" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 46253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:54:35.490156 31072 mvcc.cc:205] Tried to move back new op lower bound from 7334696040350838784 to 7334696040189579264. Current Snapshot: MvccSnapshot[applied={T|T < 7334696040350838784}]
I20260929 15:54:35.504909 30594 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:54:35.505213 30594 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:54:35.508863 30594 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e: Sending DeleteTablet for 3 replicas of tablet 86be4b087da44acba2c72d3c8e1991f7
I20260929 15:54:35.510056 30977 tablet_service.cc:1514] Processing DeleteTablet for tablet 86be4b087da44acba2c72d3c8e1991f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:54:35 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:44436
I20260929 15:54:35.510178 30846 tablet_service.cc:1514] Processing DeleteTablet for tablet 86be4b087da44acba2c72d3c8e1991f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:54:35 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:51536
I20260929 15:54:35.510785 30714 tablet_service.cc:1514] Processing DeleteTablet for tablet 86be4b087da44acba2c72d3c8e1991f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:54:35 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:49870
I20260929 15:54:35.510869 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 30631
I20260929 15:54:35.511924 31082 tablet_replica.cc:330] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7: stopping tablet replica
I20260929 15:54:35.512449 31082 raft_consensus.cc:2229] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:54:35.512980 31082 raft_consensus.cc:2258] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:54:35.513831 31080 tablet_replica.cc:330] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9: stopping tablet replica
I20260929 15:54:35.514274 31080 raft_consensus.cc:2229] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:54:35.515081 31080 raft_consensus.cc:2258] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:54:35.515916 31081 tablet_replica.cc:330] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0: stopping tablet replica
I20260929 15:54:35.516304 31081 raft_consensus.cc:2229] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:54:35.516799 31081 raft_consensus.cc:2258] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:54:35.516819 31082 ts_tablet_manager.cc:1905] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:54:35.519126 31081 ts_tablet_manager.cc:1905] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:54:35.520063 31080 ts_tablet_manager.cc:1905] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:54:35.526028 31081 ts_tablet_manager.cc:1918] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:54:35.526253 31081 log.cc:1196] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/86be4b087da44acba2c72d3c8e1991f7
I20260929 15:54:35.526991 31081 ts_tablet_manager.cc:1939] T 86be4b087da44acba2c72d3c8e1991f7 P 5f7986e698c643f69a2ab6aa83873ae0: Deleting consensus metadata
I20260929 15:54:35.529165 30580 catalog_manager.cc:4841] TS 5f7986e698c643f69a2ab6aa83873ae0 (127.15.196.194:32989): tablet 86be4b087da44acba2c72d3c8e1991f7 (table test-table [id=704e653341134a3f8d38d8ba9b359cfa]) successfully deleted
I20260929 15:54:35.530411 31055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.512015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49884 (local address 127.15.196.193:46253)
0929 15:54:35.512195 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:35.512201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:35.512220 (+ 19us) server_negotiation.cc:407] Connection header received
0929 15:54:35.512276 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:35.512282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:35.512365 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:35.512473 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:35.514334 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.515471 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.517427 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.518059 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.518218 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.521967 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:35.521984 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:35.522003 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:35.522053 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:35.526131 (+ 4078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:35.527079 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:35.527088 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:35.527092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:35.527241 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:35.527764 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:35.527770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:35.527773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:35.528232 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:35.528526 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:35.529736 (+ 1210us) server_negotiation.cc:299] Negotiation successful
0929 15:54:35.529976 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 15:54:35.531363 30774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:35.533417 31080 ts_tablet_manager.cc:1918] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:54:35.533686 31080 log.cc:1196] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/86be4b087da44acba2c72d3c8e1991f7
I20260929 15:54:35.534397 31080 ts_tablet_manager.cc:1939] T 86be4b087da44acba2c72d3c8e1991f7 P ac90d00b24f44f02a5c825ad982e17e9: Deleting consensus metadata
I20260929 15:54:35.536648 30580 catalog_manager.cc:4841] TS ac90d00b24f44f02a5c825ad982e17e9 (127.15.196.195:44415): tablet 86be4b087da44acba2c72d3c8e1991f7 (table test-table [id=704e653341134a3f8d38d8ba9b359cfa]) successfully deleted
I20260929 15:54:35.624495 31082 ts_tablet_manager.cc:1918] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:54:35.624770 31082 log.cc:1196] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/86be4b087da44acba2c72d3c8e1991f7
I20260929 15:54:35.625108 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 30631
W20260929 15:54:35.643388 30579 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.196.193:46253 (error 108)
I20260929 15:54:35.643657 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 30783
I20260929 15:54:35.645704 31052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.645300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:46253 (local address 127.0.0.1:49888)
0929 15:54:35.645469 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:35.645574 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260929 15:54:35.646023 30579 catalog_manager.cc:4584] TS d24a657d04014827a741baae1dcd09a7 (127.15.196.193:46253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 86be4b087da44acba2c72d3c8e1991f7: Network error: Client connection negotiation failed: client connection to 127.15.196.193:46253: connect: Connection refused (error 111)
I20260929 15:54:35.658303 31057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.644615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51542 (local address 127.15.196.194:32989)
0929 15:54:35.644793 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:35.644797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:35.644834 (+ 37us) server_negotiation.cc:407] Connection header received
0929 15:54:35.644891 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:35.644895 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:35.644963 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:35.645039 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:35.646802 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.647672 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.649204 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.649605 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.649749 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.652479 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:35.652491 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:35.652506 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:35.652539 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:35.655505 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:35.656200 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:35.656208 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:35.656211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:35.656328 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:35.656697 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:35.656704 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:35.656707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:35.657063 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:35.657376 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:35.657717 (+ 341us) server_negotiation.cc:299] Negotiation successful
0929 15:54:35.657932 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 15:54:35.658916 30906 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:35.699923 31052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.699357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:46253 (local address 127.0.0.1:49902)
0929 15:54:35.699592 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:35.699726 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260929 15:54:35.701166 31052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.700713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:46253 (local address 127.0.0.1:49908)
0929 15:54:35.700879 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:35.700983 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 15:54:35.750344 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 30783
I20260929 15:54:35.761926 31052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.761281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:46253 (local address 127.0.0.1:49916)
0929 15:54:35.761575 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:35.761698 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260929 15:54:35.763156 31052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.762732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:46253 (local address 127.0.0.1:49920)
0929 15:54:35.762903 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:35.762993 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260929 15:54:35.768071 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 30914
I20260929 15:54:35.782344 31053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.768821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44460 (local address 127.15.196.195:44415)
0929 15:54:35.769002 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:35.769010 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:35.769091 (+ 81us) server_negotiation.cc:407] Connection header received
0929 15:54:35.769290 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:35.769296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:35.769380 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:35.769495 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:35.770855 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.771793 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.773096 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:35.773552 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:35.773682 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:35.776402 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:35.776414 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:35.776428 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:35.776463 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:35.779500 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:35.780196 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:35.780209 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:35.780212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:35.780363 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:35.780753 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:35.780760 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:35.780776 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:35.781117 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:35.781460 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:35.781735 (+ 275us) server_negotiation.cc:299] Negotiation successful
0929 15:54:35.781954 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 15:54:35.783125 31037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:35.843835 31052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.843347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:46253 (local address 127.0.0.1:49924)
0929 15:54:35.843560 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:35.843666 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260929 15:54:35.844908 31052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:35.844502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:46253 (local address 127.0.0.1:49928)
0929 15:54:35.844733 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:35.844801 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260929 15:54:35.908174 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 30914
I20260929 15:54:35.928475 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 30561
I20260929 15:54:35.929685 30624 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:36.049283 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 30561
I20260929 15:54:36.071097 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:44615
--webserver_interface=127.15.196.254
--webserver_port=44833
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:35523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:44615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:54:36.400473 31088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:36.400779 31088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:36.400870 31088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:36.408246 31088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:54:36.408478 31088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:36.408573 31088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:36.408656 31088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:54:36.408720 31088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:54:36.418849 31088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:35523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:44615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:44615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=44833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:36.420292 31088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:36.421615 31088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:36.433734 31093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:36.433712 31094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:36.433789 31096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:36.434542 31088 server_base.cc:1034] running on GCE node
I20260929 15:54:36.435269 31088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:36.436744 31088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:36.437965 31088 hybrid_clock.cc:648] HybridClock initialized: now 1790697276437889 us; error 62 us; skew 500 ppm
I20260929 15:54:36.438351 31088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:36 dist-test-slave-c51v krb5kdc[30011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697276, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:36.441568 31088 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:54:36.443058 31088 webserver.cc:460] Webserver started at http://127.15.196.254:44833/ using document root <none> and password file <none>
I20260929 15:54:36.443782 31088 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:36.443887 31088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:36.449258 31088 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 15:54:36.452144 31103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:36.453528 31088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:54:36.453729 31088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "4d9bd3eddcd34685a24d0466f4879f6e"
format_stamp: "Formatted at 2026-09-29 15:54:30 on dist-test-slave-c51v"
server_key: "4e27288f0b67470df12cc7f7d0a2177b"
server_key_iv: "b565c6203ccd626c48ac0b46be5c832e"
server_key_version: "encryptionkey@0"
I20260929 15:54:36.454723 31088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:36.472996 31088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:36.477623 31088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:36.477917 31088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:36.496845 31088 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:44615
I20260929 15:54:36.496857 31154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:44615 every 8 connection(s)
I20260929 15:54:36.498459 31088 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:54:36.504215 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 31088
I20260929 15:54:36.505293 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:46253
--local_ip_for_outbound_sockets=127.15.196.193
--tserver_master_addrs=127.15.196.254:44615
--webserver_port=38839
--webserver_interface=127.15.196.193
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:35523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:54:36.506071 31155 sys_catalog.cc:263] Verifying existing consensus state
I20260929 15:54:36.511507 31155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e: Bootstrap starting.
I20260929 15:54:36.533226 31155 log.cc:824] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:36.549151 31155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 15:54:36.549719 31155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e: Bootstrap complete.
I20260929 15:54:36.559267 31155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } }
I20260929 15:54:36.560199 31155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9bd3eddcd34685a24d0466f4879f6e, State: Initialized, Role: FOLLOWER
I20260929 15:54:36.560811 31155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } }
I20260929 15:54:36.561039 31155 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:54:36.561156 31155 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:54:36.561314 31155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 2 FOLLOWER]: Advancing to term 3
I20260929 15:54:36.565250 31155 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } }
I20260929 15:54:36.565743 31155 leader_election.cc:302] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d9bd3eddcd34685a24d0466f4879f6e; no voters:
I20260929 15:54:36.566432 31155 leader_election.cc:288] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 15:54:36.566582 31159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 3 FOLLOWER]: Leader election won for term 3
I20260929 15:54:36.567606 31159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [term 3 LEADER]: Becoming Leader. State: Replica: 4d9bd3eddcd34685a24d0466f4879f6e, State: Running, Role: LEADER
I20260929 15:54:36.568249 31159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } }
I20260929 15:54:36.568645 31155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:54:36.570935 31160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } } }
I20260929 15:54:36.571761 31160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:36.571437 31161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d9bd3eddcd34685a24d0466f4879f6e. Latest consensus state: current_term: 3 leader_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d9bd3eddcd34685a24d0466f4879f6e" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 44615 } } }
I20260929 15:54:36.572196 31161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d9bd3eddcd34685a24d0466f4879f6e [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:36.578979 31165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:54:36.589828 31165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:54:36.591909 31165 catalog_manager.cc:1259] Loaded cluster ID: 8dfd4078f4b64f8cb4a2236da0e388ff
I20260929 15:54:36.592113 31165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260929 15:54:36.595316 31165 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0030e489ad google::LogMessage::Fail() at ??:0
@ 0x7f0030e4b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f0030e484de google::LogMessage::Flush() at ??:0
@ 0x7f0030e49a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f004687b512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f0046878976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0046900610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f0046900439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0033a94345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0033a9791c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f0033a97799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0033a72eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0036d4c6db start_thread at ??:0
@ 0x7f002fce261f clone at ??:0
W20260929 15:54:36.914530 31157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:36.914882 31157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:36.915052 31157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:36.922451 31157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:36.922590 31157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:36.922770 31157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:54:36.932756 31157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:35523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:46253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=38839
--tserver_master_addrs=127.15.196.254:44615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:36.934109 31157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:36.935527 31157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:36.951550 31195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:36.951647 31197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:36.952315 31157 server_base.cc:1034] running on GCE node
W20260929 15:54:36.952383 31194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:36.953107 31157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:36.954145 31157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:36.955392 31157 hybrid_clock.cc:648] HybridClock initialized: now 1790697276955347 us; error 50 us; skew 500 ppm
I20260929 15:54:36.955732 31157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:36 dist-test-slave-c51v krb5kdc[30011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697276, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:36.959023 31157 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:54:36.960594 31157 webserver.cc:460] Webserver started at http://127.15.196.193:38839/ using document root <none> and password file <none>
I20260929 15:54:36.961251 31157 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:36.961359 31157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:36.966773 31157 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 15:54:36.969620 31204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:36.970975 31157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 15:54:36.971153 31157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "d24a657d04014827a741baae1dcd09a7"
format_stamp: "Formatted at 2026-09-29 15:54:31 on dist-test-slave-c51v"
server_key: "acd541a656e3765dbe63472ebc5629d4"
server_key_iv: "5489a1599fcc40057dbd102d43bfbe4b"
server_key_version: "encryptionkey@0"
I20260929 15:54:36.972232 31157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:37.001137 31157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:37.005779 31157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:37.006076 31157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:37.007112 31157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:37.008960 31211 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260929 15:54:37.014400 31211 ts_tablet_manager.cc:1852] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260929 15:54:37.015549 31211 ts_tablet_manager.cc:1905] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:54:37.022876 31211 ts_tablet_manager.cc:1918] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:54:37.023110 31211 log.cc:1196] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/86be4b087da44acba2c72d3c8e1991f7
I20260929 15:54:37.023662 31211 ts_tablet_manager.cc:1939] T 86be4b087da44acba2c72d3c8e1991f7 P d24a657d04014827a741baae1dcd09a7: Deleting consensus metadata
I20260929 15:54:37.024070 31157 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260929 15:54:37.024199 31157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.016s user 0.001s sys 0.000s
I20260929 15:54:37.024358 31157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:37.024451 31157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:37.058198 31157 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:46253
I20260929 15:54:37.058250 31317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:46253 every 8 connection(s)
I20260929 15:54:37.059851 31157 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:54:37.063370 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 31157
I20260929 15:54:37.064254 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:32989
--local_ip_for_outbound_sockets=127.15.196.194
--tserver_master_addrs=127.15.196.254:44615
--webserver_port=43785
--webserver_interface=127.15.196.194
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:35523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:54:37.065167 31320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:37.063045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:48107)
0929 15:54:37.063936 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:37.064174 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":371,"threads_started":1}
W20260929 15:54:37.065795 31318 heartbeater.cc:644] Failed to heartbeat to 127.15.196.254:44615 (0 consecutive failures): Network error: Failed to ping master at 127.15.196.254:44615: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
I20260929 15:54:37.066630 31320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:37.066297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:39015)
0929 15:54:37.066450 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:37.066525 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260929 15:54:37.067580 31320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:37.067227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:36091)
0929 15:54:37.067399 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:37.067466 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260929 15:54:37.067836 31318 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 15:54:37.392874 31321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:37.393153 31321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:37.393249 31321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:37.400691 31321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:37.400806 31321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:37.400928 31321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:54:37.410813 31321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:35523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:32989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=43785
--tserver_master_addrs=127.15.196.254:44615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:37.412212 31321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:37.413571 31321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:37.427335 31327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:37.427578 31326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:37.427378 31329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:37.428560 31321 server_base.cc:1034] running on GCE node
I20260929 15:54:37.429221 31321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:37.430110 31321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:37.431380 31321 hybrid_clock.cc:648] HybridClock initialized: now 1790697277431308 us; error 52 us; skew 500 ppm
I20260929 15:54:37.431751 31321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:37 dist-test-slave-c51v krb5kdc[30011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697277, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:37.434862 31321 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:54:37.436378 31321 webserver.cc:460] Webserver started at http://127.15.196.194:43785/ using document root <none> and password file <none>
I20260929 15:54:37.437157 31321 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:37.437302 31321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:37.442737 31321 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 15:54:37.445493 31336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:37.446795 31321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:54:37.446977 31321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "5f7986e698c643f69a2ab6aa83873ae0"
format_stamp: "Formatted at 2026-09-29 15:54:31 on dist-test-slave-c51v"
server_key: "dfa55e58ba544d57fbde772c3046aba0"
server_key_iv: "6d06a957397b5c1a771a5576e2e107f7"
server_key_version: "encryptionkey@0"
I20260929 15:54:37.448058 31321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:37.471271 31321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:37.475481 31321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:37.475764 31321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:37.476884 31321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:37.478312 31321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:37.478440 31321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:37.478580 31321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:37.478667 31321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:37.512657 31321 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:32989
I20260929 15:54:37.512749 31448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:32989 every 8 connection(s)
I20260929 15:54:37.514200 31321 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:54:37.518742 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 31321
I20260929 15:54:37.519583 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:44415
--local_ip_for_outbound_sockets=127.15.196.195
--tserver_master_addrs=127.15.196.254:44615
--webserver_port=44869
--webserver_interface=127.15.196.195
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:35523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:54:37.519691 31451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:37.517382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:42367)
0929 15:54:37.518094 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:37.518421 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":268,"threads_started":1}
W20260929 15:54:37.520460 31449 heartbeater.cc:644] Failed to heartbeat to 127.15.196.254:44615 (0 consecutive failures): Network error: Failed to ping master at 127.15.196.254:44615: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
I20260929 15:54:37.521461 31451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:37.521012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:45589)
0929 15:54:37.521216 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:37.521310 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 15:54:37.522567 31451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:37.522126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:58275)
0929 15:54:37.522304 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:37.522411 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260929 15:54:37.522861 31449 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 15:54:37.844946 31452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:37.845232 31452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:37.845395 31452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:37.853042 31452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:37.853173 31452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:37.853365 31452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:54:37.863129 31452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:35523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:44415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=44869
--tserver_master_addrs=127.15.196.254:44615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:37.864495 31452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:37.865866 31452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:37.879794 31457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:37.879732 31460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:37.879771 31458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:37.880494 31452 server_base.cc:1034] running on GCE node
I20260929 15:54:37.881124 31452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:37.882035 31452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:37.883270 31452 hybrid_clock.cc:648] HybridClock initialized: now 1790697277883192 us; error 63 us; skew 500 ppm
I20260929 15:54:37.883723 31452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:37 dist-test-slave-c51v krb5kdc[30011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697277, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:37.888171 31452 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:54:37.890427 31452 webserver.cc:460] Webserver started at http://127.15.196.195:44869/ using document root <none> and password file <none>
I20260929 15:54:37.891474 31452 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:37.891634 31452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:37.898520 31452 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 15:54:37.901438 31467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:37.902921 31452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:54:37.903103 31452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "ac90d00b24f44f02a5c825ad982e17e9"
format_stamp: "Formatted at 2026-09-29 15:54:32 on dist-test-slave-c51v"
server_key: "401debde1357af855a48da25a1b4f2cc"
server_key_iv: "69761b234e707668721f1d8ee68101b7"
server_key_version: "encryptionkey@0"
I20260929 15:54:37.904145 31452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:37.934465 31452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:37.938513 31452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:37.938794 31452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:37.939867 31452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:37.941171 31452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:37.941291 31452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:37.941428 31452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:37.941519 31452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:37.974004 31452 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:44415
I20260929 15:54:37.974067 31579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:44415 every 8 connection(s)
I20260929 15:54:37.975586 31452 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:54:37.980356 31582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:37.978563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:58983)
0929 15:54:37.979256 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:37.979541 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":264,"threads_started":1}
W20260929 15:54:37.980969 31580 heartbeater.cc:644] Failed to heartbeat to 127.15.196.254:44615 (0 consecutive failures): Network error: Failed to ping master at 127.15.196.254:44615: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
I20260929 15:54:37.981737 31582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:37.981450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:36099)
0929 15:54:37.981581 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:37.981644 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 15:54:37.982517 31582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:37.982240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:38247)
0929 15:54:37.982352 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:37.982419 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260929 15:54:37.982774 31580 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 15:54:37.983037 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 31452
I20260929 15:54:38.069119 31584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:38.068352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:47271)
0929 15:54:38.068790 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:38.068887 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":207,"threads_started":1}
I20260929 15:54:38.524214 31585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:38.523433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:43241)
0929 15:54:38.523875 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:38.523992 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":209,"threads_started":1}
I20260929 15:54:38.984081 31586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:38.983268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:47759)
0929 15:54:38.983762 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:38.983873 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":232,"threads_started":1}
I20260929 15:54:39.070736 31587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:39.070072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:39817)
0929 15:54:39.070492 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:39.070587 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":223,"threads_started":1}
I20260929 15:54:39.525974 31588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:39.525104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:39959)
0929 15:54:39.525717 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:39.525806 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":352,"threads_started":1}
I20260929 15:54:39.985654 31589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:39.984936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:44161)
0929 15:54:39.985412 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:39.985498 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":225,"threads_started":1}
I20260929 15:54:40.072429 31590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:40.071712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:49231)
0929 15:54:40.072175 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:40.072260 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":228,"threads_started":1}
I20260929 15:54:40.527760 31591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:40.526958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:34767)
0929 15:54:40.527482 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:40.527568 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":256,"threads_started":1}
I20260929 15:54:40.987475 31592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:40.986668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:50569)
0929 15:54:40.987158 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:40.987234 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":287,"threads_started":1}
I20260929 15:54:41.074033 31593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:41.073328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:49499)
0929 15:54:41.073774 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:41.073874 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":232,"threads_started":1}
I20260929 15:54:41.529670 31594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:41.528800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:40547)
0929 15:54:41.529315 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:41.529398 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":287,"threads_started":1}
I20260929 15:54:41.989137 31595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:41.988364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:55425)
0929 15:54:41.988883 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:41.988971 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":300,"threads_started":1}
I20260929 15:54:42.075659 31596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:42.074907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:49253)
0929 15:54:42.075394 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:42.075485 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":222,"threads_started":1}
I20260929 15:54:42.531503 31597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:42.530650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:60715)
0929 15:54:42.531196 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:42.531275 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":302,"threads_started":1}
I20260929 15:54:42.990706 31598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:42.990022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:58903)
0929 15:54:42.990455 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:42.990531 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":236,"threads_started":1}
I20260929 15:54:43.077306 31599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:43.076585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:51961)
0929 15:54:43.077051 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:43.077149 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":254,"threads_started":1}
I20260929 15:54:43.533149 31600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:43.532421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:36453)
0929 15:54:43.532884 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:43.532979 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":230,"threads_started":1}
I20260929 15:54:43.992233 31601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:43.991561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:57053)
0929 15:54:43.991976 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:43.992077 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":210,"threads_started":1}
I20260929 15:54:44.079176 31602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:44.078218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:49061)
0929 15:54:44.078879 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:44.078957 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":388,"threads_started":1}
I20260929 15:54:44.535076 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:44.534155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:33423)
0929 15:54:44.534724 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:44.534807 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":282,"threads_started":1}
I20260929 15:54:44.993997 31604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:44.993239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:45793)
0929 15:54:44.993713 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:44.993801 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":213,"threads_started":1}
I20260929 15:54:45.080693 31605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:45.080012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:38913)
0929 15:54:45.080448 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:45.080529 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":238,"threads_started":1}
I20260929 15:54:45.536834 31606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:45.536041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:55389)
0929 15:54:45.536565 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:45.536658 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":244,"threads_started":1}
I20260929 15:54:45.995599 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:45.994845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:49149)
0929 15:54:45.995355 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:45.995444 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":223,"threads_started":1}
I20260929 15:54:46.082302 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:46.081557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:48797)
0929 15:54:46.082040 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:46.082137 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":238,"threads_started":1}
I20260929 15:54:46.538471 31609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:46.537706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:55625)
0929 15:54:46.538212 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:46.538294 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":233,"threads_started":1}
I20260929 15:54:46.997184 31610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:46.996473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:40635)
0929 15:54:46.996915 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:46.997021 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":213,"threads_started":1}
I20260929 15:54:47.083901 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:47.083190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:37949)
0929 15:54:47.083659 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:47.083737 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":260,"threads_started":1}
I20260929 15:54:47.540122 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:47.539420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:47757)
0929 15:54:47.539874 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:47.539957 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":238,"threads_started":1}
I20260929 15:54:47.998879 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:47.998111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:47755)
0929 15:54:47.998631 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:47.998722 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":267,"threads_started":1}
I20260929 15:54:48.085637 31614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:48.084923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:40989)
0929 15:54:48.085354 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:48.085431 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":243,"threads_started":1}
I20260929 15:54:48.541790 31615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:48.541098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:50789)
0929 15:54:48.541516 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:48.541615 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":212,"threads_started":1}
I20260929 15:54:49.000545 31616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:48.999853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:48111)
0929 15:54:49.000279 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:49.000371 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":227,"threads_started":1}
I20260929 15:54:49.087527 31617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:49.086626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:57997)
0929 15:54:49.087197 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:49.087339 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":256,"threads_started":1}
I20260929 15:54:49.543599 31618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:49.542813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:52375)
0929 15:54:49.543280 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:49.543442 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":250,"threads_started":1}
I20260929 15:54:50.002123 31619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:50.001437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:53785)
0929 15:54:50.001896 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:50.001975 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":250,"threads_started":1}
I20260929 15:54:50.089131 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:50.088396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:41687)
0929 15:54:50.088853 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:50.088950 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":239,"threads_started":1}
I20260929 15:54:50.545233 31621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:50.544464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:41103)
0929 15:54:50.544944 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:50.545030 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":249,"threads_started":1}
I20260929 15:54:51.003787 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:51.003026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:50867)
0929 15:54:51.003518 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:51.003606 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":282,"threads_started":1}
I20260929 15:54:51.090799 31623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:51.089983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:59371)
0929 15:54:51.090513 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:51.090594 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":311,"threads_started":1}
I20260929 15:54:51.546828 31624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:51.546127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:48413)
0929 15:54:51.546549 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:51.546644 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":218,"threads_started":1}
I20260929 15:54:52.005407 31625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:52.004694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:44973)
0929 15:54:52.005131 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:52.005218 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":217,"threads_started":1}
I20260929 15:54:52.092813 31626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:52.092072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.193:40719)
0929 15:54:52.092507 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:52.092641 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":213,"threads_started":1}
I20260929 15:54:52.548863 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:52.548001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.194:36255)
0929 15:54:52.548560 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:52.548684 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":331,"threads_started":1}
I20260929 15:54:52.984421 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 31157
I20260929 15:54:53.003794 31628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:52.985578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42846 (local address 127.15.196.193:46253)
0929 15:54:52.986254 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:52.986267 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:52.986308 (+ 41us) server_negotiation.cc:407] Connection header received
0929 15:54:52.988828 (+ 2520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:52.988839 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:52.989306 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:52.989672 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:52.991177 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:52.992722 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:52.994029 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:52.994738 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:52.994946 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:52.997648 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:52.997657 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:52.997673 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:52.997710 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:53.000431 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:53.001147 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:53.001154 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:53.001160 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:53.001315 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:53.001702 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:53.001708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:53.001712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:53.002236 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:53.002553 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:53.002995 (+ 442us) server_negotiation.cc:299] Negotiation successful
0929 15:54:53.003490 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":373,"threads_started":1}
I20260929 15:54:53.005226 31313 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:53.009508 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:53.006605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:44615 (local address 127.15.196.195:58193)
0929 15:54:53.009241 (+ 2636us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:53.009348 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:44615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2502,"thread_start_us":2389,"threads_started":1}
I20260929 15:54:53.093036 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 31157
I20260929 15:54:53.110540 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 31321
I20260929 15:54:53.127759 31631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:53.111555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50714 (local address 127.15.196.194:32989)
0929 15:54:53.112106 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:53.112122 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:53.112164 (+ 42us) server_negotiation.cc:407] Connection header received
0929 15:54:53.113767 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:53.113777 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:53.114207 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:53.114503 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:53.115922 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:53.117138 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:53.118329 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:53.118987 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:53.119115 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:53.121768 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:53.121776 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:53.121791 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:53.121827 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:53.124843 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:53.125490 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:53.125496 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:53.125502 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:53.125685 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:53.125997 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:53.126000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:53.126002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:53.126498 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:53.126746 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:53.127042 (+ 296us) server_negotiation.cc:299] Negotiation successful
0929 15:54:53.127472 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":272,"threads_started":1}
I20260929 15:54:53.129251 31444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:53.218520 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 31321
I20260929 15:54:53.236485 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 31452
I20260929 15:54:53.256011 31633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:53.237639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49912 (local address 127.15.196.195:44415)
0929 15:54:53.238387 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:53.238401 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:53.238443 (+ 42us) server_negotiation.cc:407] Connection header received
0929 15:54:53.240322 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:53.240332 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:53.240771 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:53.241192 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:53.242933 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:53.244310 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:53.245638 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:53.246362 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:53.246567 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:53.249401 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:53.249408 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:53.249425 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:53.249470 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:53.252606 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:53.253328 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:53.253336 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:53.253342 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:53.253505 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:53.253897 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:53.253902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:53.253905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:53.254446 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:53.254774 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:53.255233 (+ 459us) server_negotiation.cc:299] Negotiation successful
0929 15:54:53.255704 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":389,"threads_started":1}
I20260929 15:54:53.257622 31575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:53.340945 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 31452
2026-09-29T15:54:53Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24276 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:54:53 dist-test-slave-c51v krb5kdc[31637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:54:53 dist-test-slave-c51v krb5kdc[31637](info): set up 2 sockets
Sep 29 15:54:53 dist-test-slave-c51v krb5kdc[31637](info): commencing operation
krb5kdc: starting...
W20260929 15:54:54.457782 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:54:54 dist-test-slave-c51v krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:54:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:54:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:54:54.700152 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:38147
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:38147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:54:55.034602 31650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:55.034946 31650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:55.035112 31650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:55.043246 31650 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 15:54:55.043401 31650 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 15:54:55.043466 31650 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 15:54:55.043542 31650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:54:55.043628 31650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:55.043726 31650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:55.043815 31650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:54:55.043893 31650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:54:55.053859 31650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:38147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:38147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:55.055214 31650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:55.056658 31650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:55.070466 31655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:55.070377 31658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:55.070417 31656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:55.071827 31650 server_base.cc:1034] running on GCE node
I20260929 15:54:55.072654 31650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:55.074100 31650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:55.075385 31650 hybrid_clock.cc:648] HybridClock initialized: now 1790697295075262 us; error 34 us; skew 500 ppm
I20260929 15:54:55.075752 31650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:55 dist-test-slave-c51v krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697295, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:55.078883 31650 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:54:55.080463 31650 webserver.cc:460] Webserver started at http://127.15.196.254:40719/ using document root <none> and password file <none>
I20260929 15:54:55.081199 31650 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:55.081288 31650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:55.081658 31650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:54:55.084764 31650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "7b34a40457ba4b21b923f0c90adbde22"
format_stamp: "Formatted at 2026-09-29 15:54:55 on dist-test-slave-c51v"
server_key: "885f25c31a1d8006566a3b3e1d3ede06"
server_key_iv: "4198fa19c6b775777ebeef8545f47782"
server_key_version: "encryptionkey@0"
I20260929 15:54:55.085554 31650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "7b34a40457ba4b21b923f0c90adbde22"
format_stamp: "Formatted at 2026-09-29 15:54:55 on dist-test-slave-c51v"
server_key: "885f25c31a1d8006566a3b3e1d3ede06"
server_key_iv: "4198fa19c6b775777ebeef8545f47782"
server_key_version: "encryptionkey@0"
I20260929 15:54:55.090739 31650 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260929 15:54:55.094214 31665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:55.095541 31650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:54:55.095728 31650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "7b34a40457ba4b21b923f0c90adbde22"
format_stamp: "Formatted at 2026-09-29 15:54:55 on dist-test-slave-c51v"
server_key: "885f25c31a1d8006566a3b3e1d3ede06"
server_key_iv: "4198fa19c6b775777ebeef8545f47782"
server_key_version: "encryptionkey@0"
I20260929 15:54:55.095947 31650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:55.121119 31650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:55.125918 31650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:55.126219 31650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:55.145244 31650 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:38147
I20260929 15:54:55.145285 31716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:38147 every 8 connection(s)
I20260929 15:54:55.146636 31650 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:54:55.150353 31717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:55.153053 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 31650
I20260929 15:54:55.153290 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:54:55.154134 16147 external_mini_cluster.cc:1428] Setting key a2750fe93037aa2c7c4011143714f42c
I20260929 15:54:55.163276 31717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Bootstrap starting.
I20260929 15:54:55.167228 31717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:55.168618 31717 log.cc:824] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Log is configured to *not* fsync() on all Append() calls
Sep 29 15:54:55 dist-test-slave-c51v krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:55.171914 31717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: No bootstrap required, opened a new log
I20260929 15:54:55.178014 31720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:55.156355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59806 (local address 127.15.196.254:38147)
0929 15:54:55.157163 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:55.157174 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:55.157215 (+ 41us) server_negotiation.cc:407] Connection header received
0929 15:54:55.158866 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:55.158889 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:55.159333 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:55.159710 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:55.161545 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:55.162843 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:55.164729 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:55.165399 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:55.165528 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:55.170683 (+ 5155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:55.170690 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:55.170704 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:55.170741 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:55.173917 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:55.174784 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:55.174790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:55.174795 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:55.174963 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:55.175395 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:55.175399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:55.175401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:55.175912 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:55.176127 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:55.176600 (+ 473us) server_negotiation.cc:299] Negotiation successful
0929 15:54:55.176963 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":392,"threads_started":1}
I20260929 15:54:55.181320 31717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } }
I20260929 15:54:55.181829 31717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:55.181958 31717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b34a40457ba4b21b923f0c90adbde22, State: Initialized, Role: FOLLOWER
I20260929 15:54:55.182603 31717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } }
I20260929 15:54:55.182837 31717 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:54:55.182952 31717 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:54:55.183086 31717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:55.185458 31717 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } }
I20260929 15:54:55.186113 31717 leader_election.cc:302] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b34a40457ba4b21b923f0c90adbde22; no voters:
I20260929 15:54:55.186794 31717 leader_election.cc:288] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:54:55.186900 31722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:54:55.187687 31722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 1 LEADER]: Becoming Leader. State: Replica: 7b34a40457ba4b21b923f0c90adbde22, State: Running, Role: LEADER
I20260929 15:54:55.188320 31722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } }
I20260929 15:54:55.189229 31717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:54:55.191433 31723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b34a40457ba4b21b923f0c90adbde22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } } }
I20260929 15:54:55.191397 31724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b34a40457ba4b21b923f0c90adbde22. Latest consensus state: current_term: 1 leader_uuid: "7b34a40457ba4b21b923f0c90adbde22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } } }
I20260929 15:54:55.191848 31723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:55.192005 31724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [sys.catalog]: This master's current role is: LEADER
I20260929 15:54:55.193538 31728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:54:55.198278 31728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:54:55.207603 31728 catalog_manager.cc:1347] Generated new cluster ID: a3f0216d378c4c01b9d602b907a2d0fa
I20260929 15:54:55.207726 31728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:54:55.225029 31728 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:54:55.226302 31728 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:54:55.231969 31728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Generated new TSK 0
I20260929 15:54:55.232771 31728 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:54:55.303794 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:38147
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:54:55.643270 31743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:55.643685 31743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:55.643867 31743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:55.651368 31743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:55.651551 31743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:55.651749 31743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:54:55.661747 31743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:38147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:55.663064 31743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:55.664413 31743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:55.677605 31749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:55.677855 31751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:55.678326 31743 server_base.cc:1034] running on GCE node
W20260929 15:54:55.678485 31748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:55.679394 31743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:55.680466 31743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:55.681720 31743 hybrid_clock.cc:648] HybridClock initialized: now 1790697295681652 us; error 65 us; skew 500 ppm
I20260929 15:54:55.682066 31743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:55 dist-test-slave-c51v krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697295, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:55.685397 31743 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:54:55.686913 31743 webserver.cc:460] Webserver started at http://127.15.196.193:45069/ using document root <none> and password file <none>
I20260929 15:54:55.687745 31743 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:55.687906 31743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:55.688227 31743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:54:55.692021 31743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "cbc9bc4e27e340ae9b340a346f6041d3"
format_stamp: "Formatted at 2026-09-29 15:54:55 on dist-test-slave-c51v"
server_key: "12e760a0d248384539a4ea62e2ad6578"
server_key_iv: "58974ee33090f3c5cea8c7f940cb823d"
server_key_version: "encryptionkey@0"
I20260929 15:54:55.692904 31743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "cbc9bc4e27e340ae9b340a346f6041d3"
format_stamp: "Formatted at 2026-09-29 15:54:55 on dist-test-slave-c51v"
server_key: "12e760a0d248384539a4ea62e2ad6578"
server_key_iv: "58974ee33090f3c5cea8c7f940cb823d"
server_key_version: "encryptionkey@0"
I20260929 15:54:55.698459 31743 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 15:54:55.702461 31758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:55.703881 31743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:54:55.704074 31743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "cbc9bc4e27e340ae9b340a346f6041d3"
format_stamp: "Formatted at 2026-09-29 15:54:55 on dist-test-slave-c51v"
server_key: "12e760a0d248384539a4ea62e2ad6578"
server_key_iv: "58974ee33090f3c5cea8c7f940cb823d"
server_key_version: "encryptionkey@0"
I20260929 15:54:55.704293 31743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:55.728103 31743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:55.732388 31743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:55.732704 31743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:55.733678 31743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:55.735016 31743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:55.735121 31743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:55.735270 31743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:55.735373 31743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260929 15:54:55.768905 31743 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:36875
I20260929 15:54:55.768958 31870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:36875 every 8 connection(s)
I20260929 15:54:55.770262 31743 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:54:55.780684 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 31743
I20260929 15:54:55.780912 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:54:55.781679 16147 external_mini_cluster.cc:1428] Setting key 38cd4a8af862126f138ec048c8874f52
Sep 29 15:54:55 dist-test-slave-c51v krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697295, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:55.791936 31873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:55.772975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:54343 (local address 127.15.196.254:38147)
0929 15:54:55.773420 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:55.773425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:55.774440 (+ 1015us) server_negotiation.cc:407] Connection header received
0929 15:54:55.776427 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:55.776434 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:55.776531 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:55.776653 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:55.778212 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:55.778937 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:55.781028 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:55.781473 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:55.781584 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:55.785674 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:55.785680 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:55.785683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:55.785715 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:55.788328 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:55.789092 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:55.789104 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:55.789107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:55.789224 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:55.789739 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:55.789747 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:55.789753 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:55.790150 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:55.790305 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:55.791151 (+ 846us) server_negotiation.cc:299] Negotiation successful
0929 15:54:55.791495 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":221,"threads_started":1}
I20260929 15:54:55.792621 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:55.773579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:54343)
0929 15:54:55.774276 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:55.774315 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:55.776147 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:55.776900 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:55.776908 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:55.777434 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:55.778021 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:55.778038 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:55.779108 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:55.779113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:55.780817 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:55.780834 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:55.781727 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:55.781738 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:55.782045 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:55.782904 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:55.782935 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:55.785418 (+ 2483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:55.788488 (+ 3070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:55.788498 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:55.788517 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:55.788956 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:55.789340 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:55.789345 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:55.789349 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:55.789545 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:55.790284 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:55.790289 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:55.790616 (+ 327us) client_negotiation.cc:769] Sending connection context
0929 15:54:55.790981 (+ 365us) client_negotiation.cc:240] Negotiation successful
0929 15:54:55.791469 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":276,"threads_started":1}
I20260929 15:54:55.794337 31871 heartbeater.cc:347] Connected to a master server at 127.15.196.254:38147
I20260929 15:54:55.794672 31871 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:55.795593 31871 heartbeater.cc:511] Master 127.15.196.254:38147 requested a full tablet report, sending...
I20260929 15:54:55.797910 31682 ts_manager.cc:194] Registered new tserver with Master: cbc9bc4e27e340ae9b340a346f6041d3 (127.15.196.193:36875)
I20260929 15:54:55.799499 31682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:54343
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:54:55.836861 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:38147
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:54:56.170914 31877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:56.171394 31877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:56.171520 31877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:56.178920 31877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:56.179033 31877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:56.179150 31877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:54:56.189260 31877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:38147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:56.190706 31877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:56.192193 31877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:56.205590 31883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:56.205561 31882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:56.205744 31885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:56.206393 31877 server_base.cc:1034] running on GCE node
I20260929 15:54:56.207037 31877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:56.208104 31877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:56.209332 31877 hybrid_clock.cc:648] HybridClock initialized: now 1790697296209280 us; error 43 us; skew 500 ppm
I20260929 15:54:56.209664 31877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:56 dist-test-slave-c51v krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697296, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:56.212791 31877 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:54:56.214162 31877 webserver.cc:460] Webserver started at http://127.15.196.194:46195/ using document root <none> and password file <none>
I20260929 15:54:56.214844 31877 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:56.214993 31877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:56.215349 31877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:54:56.218240 31877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "fa4812e4098c494994d66a14d5da2638"
format_stamp: "Formatted at 2026-09-29 15:54:56 on dist-test-slave-c51v"
server_key: "9733bfb2c26afbd45e740241bd6bfc64"
server_key_iv: "03990abe4eca8d470c210b6f2422b5f3"
server_key_version: "encryptionkey@0"
I20260929 15:54:56.218995 31877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "fa4812e4098c494994d66a14d5da2638"
format_stamp: "Formatted at 2026-09-29 15:54:56 on dist-test-slave-c51v"
server_key: "9733bfb2c26afbd45e740241bd6bfc64"
server_key_iv: "03990abe4eca8d470c210b6f2422b5f3"
server_key_version: "encryptionkey@0"
I20260929 15:54:56.224148 31877 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 15:54:56.228116 31892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:56.229705 31877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:54:56.230010 31877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "fa4812e4098c494994d66a14d5da2638"
format_stamp: "Formatted at 2026-09-29 15:54:56 on dist-test-slave-c51v"
server_key: "9733bfb2c26afbd45e740241bd6bfc64"
server_key_iv: "03990abe4eca8d470c210b6f2422b5f3"
server_key_version: "encryptionkey@0"
I20260929 15:54:56.230228 31877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:56.247227 31877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:56.251916 31877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:56.252283 31877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:56.253314 31877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:56.254704 31877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:56.254810 31877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:56.254984 31877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:56.255072 31877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:56.289664 31877 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:33615
I20260929 15:54:56.289812 32004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:33615 every 8 connection(s)
I20260929 15:54:56.291046 31877 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:54:56.300194 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 31877
I20260929 15:54:56.300405 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:54:56.300931 16147 external_mini_cluster.cc:1428] Setting key bd199598e840d1fe745e286b9741d64e
Sep 29 15:54:56 dist-test-slave-c51v krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697296, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:56.313980 32007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:56.293992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:35009 (local address 127.15.196.254:38147)
0929 15:54:56.294432 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:56.294437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:56.295396 (+ 959us) server_negotiation.cc:407] Connection header received
0929 15:54:56.297219 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:56.297225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:56.297321 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:56.297444 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:56.298981 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.299814 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.301993 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.302632 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.302761 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:56.306940 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:56.306949 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:56.306956 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:56.307001 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:56.310754 (+ 3753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:56.311558 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:56.311566 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:56.311572 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:56.311741 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:56.312156 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:56.312159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:56.312161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:56.312422 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:56.312543 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:56.313518 (+ 975us) server_negotiation.cc:299] Negotiation successful
0929 15:54:56.313702 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":237,"threads_started":1}
I20260929 15:54:56.315493 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:56.294502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:35009)
0929 15:54:56.295200 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:56.295238 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:56.296941 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:56.297690 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:56.297699 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:56.298216 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:56.298805 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:56.298822 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.299987 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.299991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:56.301815 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:56.301828 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.302869 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.302877 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:56.303177 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:56.303970 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:56.304002 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:56.306709 (+ 2707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:56.310904 (+ 4195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:56.310911 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:56.310925 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:56.311442 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:56.311851 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:56.311854 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:56.311857 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:56.312052 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:56.312571 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:56.312582 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:56.313058 (+ 476us) client_negotiation.cc:769] Sending connection context
0929 15:54:56.313384 (+ 326us) client_negotiation.cc:240] Negotiation successful
0929 15:54:56.313985 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":265,"threads_started":1}
I20260929 15:54:56.317023 32005 heartbeater.cc:347] Connected to a master server at 127.15.196.254:38147
I20260929 15:54:56.317335 32005 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:56.318220 32005 heartbeater.cc:511] Master 127.15.196.254:38147 requested a full tablet report, sending...
I20260929 15:54:56.319962 31682 ts_manager.cc:194] Registered new tserver with Master: fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615)
I20260929 15:54:56.321244 31682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:35009
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:54:56.358623 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:38147
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:54:56.686205 32011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:54:56.686488 32011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:54:56.686586 32011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:54:56.694021 32011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:54:56.694144 32011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:54:56.694258 32011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:54:56.704315 32011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:38147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:54:56.705569 32011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:54:56.706816 32011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:54:56.720196 32017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:56.720265 32016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:54:56.720266 32019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:54:56.720894 32011 server_base.cc:1034] running on GCE node
I20260929 15:54:56.721475 32011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:54:56.722378 32011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:54:56.723595 32011 hybrid_clock.cc:648] HybridClock initialized: now 1790697296723533 us; error 59 us; skew 500 ppm
I20260929 15:54:56.723904 32011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:54:56 dist-test-slave-c51v krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697296, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:54:56.726991 32011 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:54:56.728540 32011 webserver.cc:460] Webserver started at http://127.15.196.195:42309/ using document root <none> and password file <none>
I20260929 15:54:56.729203 32011 fs_manager.cc:359] Metadata directory not provided
I20260929 15:54:56.729346 32011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:54:56.729637 32011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:54:56.732488 32011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "f84e04a0d55747e2a5d2a6bb4bc77147"
format_stamp: "Formatted at 2026-09-29 15:54:56 on dist-test-slave-c51v"
server_key: "809f49e30db4d3833b1e0bc29b21a61d"
server_key_iv: "948fcfe13082a92522cf93e481e08532"
server_key_version: "encryptionkey@0"
I20260929 15:54:56.733428 32011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "f84e04a0d55747e2a5d2a6bb4bc77147"
format_stamp: "Formatted at 2026-09-29 15:54:56 on dist-test-slave-c51v"
server_key: "809f49e30db4d3833b1e0bc29b21a61d"
server_key_iv: "948fcfe13082a92522cf93e481e08532"
server_key_version: "encryptionkey@0"
I20260929 15:54:56.738734 32011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 15:54:56.742139 32026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:56.743527 32011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:54:56.743701 32011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "f84e04a0d55747e2a5d2a6bb4bc77147"
format_stamp: "Formatted at 2026-09-29 15:54:56 on dist-test-slave-c51v"
server_key: "809f49e30db4d3833b1e0bc29b21a61d"
server_key_iv: "948fcfe13082a92522cf93e481e08532"
server_key_version: "encryptionkey@0"
I20260929 15:54:56.743901 32011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:54:56.797827 32011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:54:56.802475 32011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:54:56.802772 32011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:54:56.803674 31871 heartbeater.cc:503] Master 127.15.196.254:38147 was elected leader, sending a full tablet report...
I20260929 15:54:56.803927 32011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:54:56.805269 32011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:54:56.805370 32011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:56.805516 32011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:54:56.805593 32011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:54:56.840134 32011 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:38143
I20260929 15:54:56.840224 32138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:38143 every 8 connection(s)
I20260929 15:54:56.841465 32011 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:54:56.851524 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 32011
I20260929 15:54:56.851716 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:54:56.852219 16147 external_mini_cluster.cc:1428] Setting key aab563c9279ef9a9113421e8b10b8c37
Sep 29 15:54:56 dist-test-slave-c51v krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697296, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:54:56.863430 32141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:56.844133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:39081 (local address 127.15.196.254:38147)
0929 15:54:56.844645 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:56.844650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:56.845792 (+ 1142us) server_negotiation.cc:407] Connection header received
0929 15:54:56.847875 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:56.847881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:56.848008 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:56.848155 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:56.849800 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.850680 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.852678 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.853269 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.853419 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:56.857399 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:56.857406 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:56.857418 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:56.857455 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:56.860120 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:56.860898 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:56.860904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:56.860910 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:56.861047 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:56.861521 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:56.861534 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:56.861536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:56.861813 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:56.862005 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:56.862727 (+ 722us) server_negotiation.cc:299] Negotiation successful
0929 15:54:56.862976 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":230,"threads_started":1}
I20260929 15:54:56.864426 32142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:56.844727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:39081)
0929 15:54:56.845596 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:56.845642 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:56.847571 (+ 1929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:56.848422 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:56.848431 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:56.848980 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:54:56.849608 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:56.849626 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.850852 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.850858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:56.852520 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:56.852532 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.853536 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.853560 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:56.853838 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:56.854607 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:54:56.854631 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:54:56.857181 (+ 2550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:54:56.860274 (+ 3093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:56.860284 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:56.860302 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:56.860759 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:56.861169 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:54:56.861176 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:54:56.861180 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:54:56.861384 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:54:56.861959 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:54:56.861968 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:54:56.862331 (+ 363us) client_negotiation.cc:769] Sending connection context
0929 15:54:56.862615 (+ 284us) client_negotiation.cc:240] Negotiation successful
0929 15:54:56.863194 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":376,"threads_started":1}
I20260929 15:54:56.865866 32139 heartbeater.cc:347] Connected to a master server at 127.15.196.254:38147
I20260929 15:54:56.866262 32139 heartbeater.cc:464] Registering TS with master...
I20260929 15:54:56.867156 32139 heartbeater.cc:511] Master 127.15.196.254:38147 requested a full tablet report, sending...
I20260929 15:54:56.868626 31682 ts_manager.cc:194] Registered new tserver with Master: f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143)
I20260929 15:54:56.869379 31682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:39081
I20260929 15:54:56.880008 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:54:56.898011 32141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:56.884687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59822 (local address 127.15.196.254:38147)
0929 15:54:56.884903 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:56.884911 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:56.885324 (+ 413us) server_negotiation.cc:407] Connection header received
0929 15:54:56.885457 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:56.885460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:56.885535 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:56.885617 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:56.887038 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.887823 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.889243 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.889689 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.889781 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:56.892322 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:56.892327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:56.892330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:56.892361 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:56.894735 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:56.895428 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:56.895435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:56.895439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:56.895574 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:56.895967 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:56.895973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:56.895976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:56.896440 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:56.896750 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:56.897408 (+ 658us) server_negotiation.cc:299] Negotiation successful
0929 15:54:56.897663 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 15:54:56.902499 31682 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:54:56.909528 31682 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:54:56.934092 32154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:56.921645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:36875 (local address 127.0.0.1:59816)
0929 15:54:56.923799 (+ 2154us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:56.923819 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:56.923938 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:56.925449 (+ 1511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:56.925457 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:56.925485 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:56.926007 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:56.926023 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.927999 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.928003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:56.930622 (+ 2619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:56.930635 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.932888 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.932900 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:56.933076 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:56.933127 (+ 51us) client_negotiation.cc:769] Sending connection context
0929 15:54:56.933316 (+ 189us) client_negotiation.cc:240] Negotiation successful
0929 15:54:56.933539 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2027,"mutex_wait_us":110,"thread_start_us":1865,"threads_started":1}
I20260929 15:54:56.935326 32153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:56.921567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41478 (local address 127.15.196.194:33615)
0929 15:54:56.925120 (+ 3553us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:56.925140 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:56.925164 (+ 24us) server_negotiation.cc:407] Connection header received
0929 15:54:56.925256 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:56.925264 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:56.925486 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:56.925730 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:56.928058 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.929910 (+ 1852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.932347 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.933676 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.934594 (+ 918us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:56.934744 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:56.934865 (+ 121us) server_negotiation.cc:299] Negotiation successful
0929 15:54:56.935018 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3406,"thread_start_us":2254,"threads_started":1}
I20260929 15:54:56.934824 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:56.923541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:38143 (local address 127.0.0.1:51384)
0929 15:54:56.924459 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:56.924476 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:56.924579 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:56.925719 (+ 1140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:56.925724 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:56.925754 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:56.926449 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:56.926458 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.928307 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.928312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:56.931657 (+ 3345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:56.931672 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.933558 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.933564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:56.933750 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:56.933783 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 15:54:56.933855 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 15:54:56.934003 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":61,"thread_start_us":616,"threads_started":1}
I20260929 15:54:56.935093 32152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:56.921356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:33615 (local address 127.0.0.1:41478)
0929 15:54:56.922200 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:56.922248 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:56.922459 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:56.925711 (+ 3252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:56.925715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:56.925742 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:56.926441 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:56.926453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.930074 (+ 3621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.930079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:56.932152 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:56.932167 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.933944 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.933950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:56.934121 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:56.934152 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 15:54:56.934211 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 15:54:56.934326 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":31,"thread_start_us":767,"threads_started":1}
I20260929 15:54:56.936242 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:56.924262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51384 (local address 127.15.196.195:38143)
0929 15:54:56.925120 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:56.925128 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:56.925159 (+ 31us) server_negotiation.cc:407] Connection header received
0929 15:54:56.925256 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:56.925264 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:56.925486 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:56.926092 (+ 606us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:56.926664 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.928147 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.931847 (+ 3700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.933353 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.934281 (+ 928us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:56.935519 (+ 1238us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:56.935592 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 15:54:56.935890 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":534,"threads_started":1}
I20260929 15:54:56.936259 32155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:56.921871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59816 (local address 127.15.196.193:36875)
0929 15:54:56.924907 (+ 3036us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:56.924916 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:56.924943 (+ 27us) server_negotiation.cc:407] Connection header received
0929 15:54:56.925052 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:56.925058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:56.925246 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:56.925456 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:56.926212 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.927868 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.930818 (+ 2950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:56.932361 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:56.932543 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:56.932738 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:56.935798 (+ 3060us) server_negotiation.cc:299] Negotiation successful
0929 15:54:56.935950 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2841,"thread_start_us":2819,"threads_started":1}
I20260929 15:54:56.940699 32074 tablet_service.cc:1467] Processing CreateTablet for tablet d165451d1f7a4f90afe44f031e00ea85 (DEFAULT_TABLE table=test-table [id=9bb2bc47c733443182cfb7510aadfa5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:54:56.941449 31806 tablet_service.cc:1467] Processing CreateTablet for tablet d165451d1f7a4f90afe44f031e00ea85 (DEFAULT_TABLE table=test-table [id=9bb2bc47c733443182cfb7510aadfa5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:54:56.942340 32074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d165451d1f7a4f90afe44f031e00ea85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:56.942978 31806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d165451d1f7a4f90afe44f031e00ea85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:56.946051 31940 tablet_service.cc:1467] Processing CreateTablet for tablet d165451d1f7a4f90afe44f031e00ea85 (DEFAULT_TABLE table=test-table [id=9bb2bc47c733443182cfb7510aadfa5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:54:56.947780 31940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d165451d1f7a4f90afe44f031e00ea85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:54:56.957525 32158 tablet_bootstrap.cc:492] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147: Bootstrap starting.
I20260929 15:54:56.960114 32159 tablet_bootstrap.cc:492] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3: Bootstrap starting.
I20260929 15:54:56.961398 32158 tablet_bootstrap.cc:654] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:56.963070 32158 log.cc:824] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:56.963166 32159 tablet_bootstrap.cc:654] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:56.964391 32159 log.cc:824] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:56.965759 32160 tablet_bootstrap.cc:492] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638: Bootstrap starting.
I20260929 15:54:56.966387 32158 tablet_bootstrap.cc:492] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147: No bootstrap required, opened a new log
I20260929 15:54:56.966650 32158 ts_tablet_manager.cc:1397] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260929 15:54:56.967031 32159 tablet_bootstrap.cc:492] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3: No bootstrap required, opened a new log
I20260929 15:54:56.967274 32159 ts_tablet_manager.cc:1397] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 15:54:56.969017 32160 tablet_bootstrap.cc:654] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638: Neither blocks nor log segments found. Creating new log.
I20260929 15:54:56.970151 32160 log.cc:824] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638: Log is configured to *not* fsync() on all Append() calls
I20260929 15:54:56.973096 32160 tablet_bootstrap.cc:492] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638: No bootstrap required, opened a new log
I20260929 15:54:56.973357 32160 ts_tablet_manager.cc:1397] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260929 15:54:56.975111 32159 raft_consensus.cc:348] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } } peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } }
I20260929 15:54:56.975529 32159 raft_consensus.cc:374] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:56.975656 32159 raft_consensus.cc:729] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbc9bc4e27e340ae9b340a346f6041d3, State: Initialized, Role: FOLLOWER
I20260929 15:54:56.976282 32159 consensus_queue.cc:260] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } } peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } }
I20260929 15:54:56.977023 32158 raft_consensus.cc:348] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } } peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } }
I20260929 15:54:56.977494 32158 raft_consensus.cc:374] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:56.977634 32158 raft_consensus.cc:729] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f84e04a0d55747e2a5d2a6bb4bc77147, State: Initialized, Role: FOLLOWER
I20260929 15:54:56.978005 32159 ts_tablet_manager.cc:1428] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3: Time spent starting tablet: real 0.011s user 0.003s sys 0.007s
I20260929 15:54:56.978406 32158 consensus_queue.cc:260] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } } peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } }
I20260929 15:54:56.980021 32139 heartbeater.cc:503] Master 127.15.196.254:38147 was elected leader, sending a full tablet report...
I20260929 15:54:56.982038 32158 ts_tablet_manager.cc:1428] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260929 15:54:56.984012 32160 raft_consensus.cc:348] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } } peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } }
I20260929 15:54:56.984452 32160 raft_consensus.cc:374] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:54:56.984570 32160 raft_consensus.cc:729] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4812e4098c494994d66a14d5da2638, State: Initialized, Role: FOLLOWER
I20260929 15:54:56.985085 32160 consensus_queue.cc:260] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } } peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } }
I20260929 15:54:56.986195 32005 heartbeater.cc:503] Master 127.15.196.254:38147 was elected leader, sending a full tablet report...
I20260929 15:54:56.986691 32160 ts_tablet_manager.cc:1428] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260929 15:54:57.022746 31872 tablet.cc:2369] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 15:54:57.043275 32006 tablet.cc:2369] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 15:54:57.093151 32140 tablet.cc:2369] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:54:57.246258 32164 raft_consensus.cc:482] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:54:57.246589 32164 raft_consensus.cc:504] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } } peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } }
I20260929 15:54:57.248683 32164 leader_election.cc:288] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615), f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143)
I20260929 15:54:57.255573 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:57.249170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:57117 (local address 127.15.196.195:38143)
0929 15:54:57.249394 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:57.249402 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:57.250010 (+ 608us) server_negotiation.cc:407] Connection header received
0929 15:54:57.250223 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:57.250231 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:57.250316 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:57.250399 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:57.251131 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:57.251949 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:57.253824 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:57.254578 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:57.254773 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:57.254869 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:57.255118 (+ 249us) server_negotiation.cc:299] Negotiation successful
0929 15:54:57.255279 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260929 15:54:57.255615 32153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:57.249079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:39831 (local address 127.15.196.194:33615)
0929 15:54:57.249311 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:57.249319 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:57.250169 (+ 850us) server_negotiation.cc:407] Connection header received
0929 15:54:57.250333 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:57.250336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:57.250427 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:57.250515 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:54:57.251099 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:57.251857 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:57.253763 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:57.254533 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:57.254689 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:57.254780 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:57.255174 (+ 394us) server_negotiation.cc:299] Negotiation successful
0929 15:54:57.255341 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 15:54:57.255838 32167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:57.249149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:38143 (local address 127.15.196.193:57117)
0929 15:54:57.249891 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:57.249907 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:57.250039 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:57.250453 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:57.250461 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:57.250491 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:57.250961 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:57.250972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:57.252091 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:57.252095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:57.253653 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:57.253664 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:57.254840 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:57.254845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:57.254963 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:57.254986 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 15:54:57.255061 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 15:54:57.255353 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"mutex_wait_us":149,"spinlock_wait_cycles":56576,"thread_start_us":388,"threads_started":1}
I20260929 15:54:57.256102 32168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:57.249183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:33615 (local address 127.15.196.193:39831)
0929 15:54:57.250025 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0929 15:54:57.250043 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:54:57.250200 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:54:57.250566 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:54:57.250575 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:54:57.250606 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:54:57.250936 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:57.250949 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:57.252001 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:57.252009 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:57.253610 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:54:57.253621 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:54:57.254813 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:54:57.254820 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:54:57.254985 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:57.255008 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 15:54:57.255127 (+ 119us) client_negotiation.cc:240] Negotiation successful
0929 15:54:57.255365 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":66,"thread_start_us":472,"threads_started":1}
I20260929 15:54:57.257007 32094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d165451d1f7a4f90afe44f031e00ea85" candidate_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" is_pre_election: true
I20260929 15:54:57.257055 31960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d165451d1f7a4f90afe44f031e00ea85" candidate_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4812e4098c494994d66a14d5da2638" is_pre_election: true
I20260929 15:54:57.257576 32094 raft_consensus.cc:2454] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbc9bc4e27e340ae9b340a346f6041d3 in term 0.
I20260929 15:54:57.257577 31960 raft_consensus.cc:2454] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbc9bc4e27e340ae9b340a346f6041d3 in term 0.
I20260929 15:54:57.258235 31761 leader_election.cc:302] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: cbc9bc4e27e340ae9b340a346f6041d3, f84e04a0d55747e2a5d2a6bb4bc77147, fa4812e4098c494994d66a14d5da2638; no voters:
I20260929 15:54:57.258671 32164 raft_consensus.cc:2781] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:54:57.258828 32164 raft_consensus.cc:482] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:54:57.258946 32164 raft_consensus.cc:3037] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:57.261189 32164 raft_consensus.cc:504] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } } peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } }
I20260929 15:54:57.261888 32164 leader_election.cc:288] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 1 election: Requested vote from peers fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615), f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143)
I20260929 15:54:57.262343 31960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d165451d1f7a4f90afe44f031e00ea85" candidate_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4812e4098c494994d66a14d5da2638"
I20260929 15:54:57.262511 32094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d165451d1f7a4f90afe44f031e00ea85" candidate_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147"
I20260929 15:54:57.262605 31960 raft_consensus.cc:3037] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:57.262777 32094 raft_consensus.cc:3037] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:54:57.264946 31960 raft_consensus.cc:2454] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbc9bc4e27e340ae9b340a346f6041d3 in term 1.
I20260929 15:54:57.265233 32094 raft_consensus.cc:2454] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbc9bc4e27e340ae9b340a346f6041d3 in term 1.
I20260929 15:54:57.265406 31762 leader_election.cc:302] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbc9bc4e27e340ae9b340a346f6041d3, fa4812e4098c494994d66a14d5da2638; no voters:
I20260929 15:54:57.265763 32164 raft_consensus.cc:2781] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:54:57.266239 32164 raft_consensus.cc:686] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 LEADER]: Becoming Leader. State: Replica: cbc9bc4e27e340ae9b340a346f6041d3, State: Running, Role: LEADER
I20260929 15:54:57.266742 32164 consensus_queue.cc:237] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } } peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } }
I20260929 15:54:57.271370 31682 catalog_manager.cc:5495] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to cbc9bc4e27e340ae9b340a346f6041d3 (127.15.196.193). New cstate: current_term: 1 leader_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } health_report { overall_health: UNKNOWN } } }
I20260929 15:54:57.299469 32155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:57.293931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59818 (local address 127.15.196.193:36875)
0929 15:54:57.294179 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:57.294187 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:57.294206 (+ 19us) server_negotiation.cc:407] Connection header received
0929 15:54:57.294336 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:57.294341 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:57.294458 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:57.294611 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:54:57.295008 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:57.295847 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:57.297268 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:57.297675 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:57.297811 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:57.298336 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:54:57.298823 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:54:57.298978 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:57.299061 (+ 83us) server_negotiation.cc:299] Negotiation successful
0929 15:54:57.299150 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 15:54:57.307646 31960 raft_consensus.cc:1261] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638 [term 1 FOLLOWER]: Refusing update from remote peer cbc9bc4e27e340ae9b340a346f6041d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:54:57.307669 32094 raft_consensus.cc:1261] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147 [term 1 FOLLOWER]: Refusing update from remote peer cbc9bc4e27e340ae9b340a346f6041d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:54:57.308600 32169 consensus_queue.cc:1035] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:54:57.309005 32164 consensus_queue.cc:1035] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:54:57.319919 32172 mvcc.cc:205] Tried to move back new op lower bound from 7334696129762590720 to 7334696129608859648. Current Snapshot: MvccSnapshot[applied={T|T < 7334696129762590720}]
I20260929 15:54:57.332623 31681 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:54:57.332890 31681 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:54:57.336658 31681 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Sending DeleteTablet for 3 replicas of tablet d165451d1f7a4f90afe44f031e00ea85
I20260929 15:54:57.337793 31940 tablet_service.cc:1514] Processing DeleteTablet for tablet d165451d1f7a4f90afe44f031e00ea85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:54:57 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:41478
I20260929 15:54:57.337869 31806 tablet_service.cc:1514] Processing DeleteTablet for tablet d165451d1f7a4f90afe44f031e00ea85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:54:57 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:59816
I20260929 15:54:57.338167 32074 tablet_service.cc:1514] Processing DeleteTablet for tablet d165451d1f7a4f90afe44f031e00ea85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:54:57 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:51384
I20260929 15:54:57.339382 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 31743
I20260929 15:54:57.349237 32184 tablet_replica.cc:330] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147: stopping tablet replica
I20260929 15:54:57.350492 32184 raft_consensus.cc:2229] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:54:57.351894 32184 raft_consensus.cc:2258] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:54:57.354943 32184 ts_tablet_manager.cc:1905] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:54:57.360935 32182 tablet_replica.cc:330] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638: stopping tablet replica
I20260929 15:54:57.361090 32183 tablet_replica.cc:330] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3: stopping tablet replica
I20260929 15:54:57.361452 32182 raft_consensus.cc:2229] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:54:57.361500 32183 raft_consensus.cc:2229] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:54:57.362041 32182 raft_consensus.cc:2258] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:54:57.362255 32183 raft_consensus.cc:2258] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:54:57.364585 32184 ts_tablet_manager.cc:1918] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:54:57.364756 32184 log.cc:1196] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/d165451d1f7a4f90afe44f031e00ea85
I20260929 15:54:57.364825 32182 ts_tablet_manager.cc:1905] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:54:57.365306 32184 ts_tablet_manager.cc:1939] T d165451d1f7a4f90afe44f031e00ea85 P f84e04a0d55747e2a5d2a6bb4bc77147: Deleting consensus metadata
I20260929 15:54:57.367197 31668 catalog_manager.cc:4841] TS f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143): tablet d165451d1f7a4f90afe44f031e00ea85 (table test-table [id=9bb2bc47c733443182cfb7510aadfa5e]) successfully deleted
I20260929 15:54:57.368269 32183 ts_tablet_manager.cc:1905] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:54:57.370594 32182 ts_tablet_manager.cc:1918] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:54:57.370829 32182 log.cc:1196] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/d165451d1f7a4f90afe44f031e00ea85
I20260929 15:54:57.371480 32182 ts_tablet_manager.cc:1939] T d165451d1f7a4f90afe44f031e00ea85 P fa4812e4098c494994d66a14d5da2638: Deleting consensus metadata
Sep 29 15:54:57 dist-test-slave-c51v krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:54:57.373354 31669 catalog_manager.cc:4841] TS fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615): tablet d165451d1f7a4f90afe44f031e00ea85 (table test-table [id=9bb2bc47c733443182cfb7510aadfa5e]) successfully deleted
I20260929 15:54:57.375437 32183 ts_tablet_manager.cc:1918] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:54:57.375628 32183 log.cc:1196] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/d165451d1f7a4f90afe44f031e00ea85
I20260929 15:54:57.376194 32183 ts_tablet_manager.cc:1939] T d165451d1f7a4f90afe44f031e00ea85 P cbc9bc4e27e340ae9b340a346f6041d3: Deleting consensus metadata
I20260929 15:54:57.378036 31666 catalog_manager.cc:4841] TS cbc9bc4e27e340ae9b340a346f6041d3 (127.15.196.193:36875): tablet d165451d1f7a4f90afe44f031e00ea85 (table test-table [id=9bb2bc47c733443182cfb7510aadfa5e]) successfully deleted
I20260929 15:54:57.380929 32155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:57.340734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59832 (local address 127.15.196.193:36875)
0929 15:54:57.340954 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:57.340963 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:57.354078 (+ 13115us) server_negotiation.cc:407] Connection header received
0929 15:54:57.354593 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:57.354599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:57.354714 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:57.354790 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:57.357076 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:57.358275 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:57.362767 (+ 4492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:57.363557 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:57.363685 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:57.374376 (+ 10691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:57.374387 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:57.374404 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:57.374457 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:57.377722 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:57.378658 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:57.378666 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:57.378668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:57.378808 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:57.379235 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:57.379239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:57.379241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:57.379678 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:57.379997 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:57.380343 (+ 346us) server_negotiation.cc:299] Negotiation successful
0929 15:54:57.380575 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 15:54:57.381707 31866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:57.482160 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 31743
I20260929 15:54:57.500804 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 31877
Sep 29 15:54:57 dist-test-slave-c51v krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:54:57.515404 32153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:57.501650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41494 (local address 127.15.196.194:33615)
0929 15:54:57.502016 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:57.502022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:57.502048 (+ 26us) server_negotiation.cc:407] Connection header received
0929 15:54:57.502117 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:57.502121 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:57.502180 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:57.502261 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:57.503652 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:57.504361 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:57.505599 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:57.505964 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:57.506054 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:57.510226 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:57.510241 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:57.510257 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:57.510294 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:57.513000 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:57.513628 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:57.513635 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:57.513638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:57.513765 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:57.514071 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:57.514075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:57.514077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:57.514437 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:57.514659 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:57.514906 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 15:54:57.515063 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260929 15:54:57.516073 32000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:57.606427 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 31877
I20260929 15:54:57.623337 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 32011
Sep 29 15:54:57 dist-test-slave-c51v krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:54:57.638664 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:54:57.624201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51388 (local address 127.15.196.195:38143)
0929 15:54:57.624409 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 15:54:57.624415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:54:57.624436 (+ 21us) server_negotiation.cc:407] Connection header received
0929 15:54:57.624625 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:54:57.624630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:54:57.624704 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:54:57.624811 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:54:57.626123 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:57.626849 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:57.628253 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:54:57.628647 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:54:57.628779 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:54:57.633130 (+ 4351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:54:57.633145 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:54:57.633162 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:54:57.633196 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:54:57.635814 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:57.636520 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:57.636528 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:57.636531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:57.636670 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:54:57.637056 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:54:57.637061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:54:57.637064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:54:57.637424 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:54:57.637733 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0929 15:54:57.638051 (+ 318us) server_negotiation.cc:299] Negotiation successful
0929 15:54:57.638311 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 15:54:57.639467 32134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:57.729702 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 32011
I20260929 15:54:57.746852 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 31650
I20260929 15:54:57.748093 31712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:54:57.860716 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 31650
I20260929 15:55:17.882902 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:38147
--webserver_interface=127.15.196.254
--webserver_port=40719
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:38147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:55:18.218976 32190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:55:18.219260 32190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:55:18.219445 32190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:55:18.226549 32190 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 15:55:18.226642 32190 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 15:55:18.226743 32190 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 15:55:18.226846 32190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:55:18.226939 32190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:55:18.227061 32190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:55:18.227139 32190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:55:18.227224 32190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:55:18.237063 32190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:38147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:38147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=40719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:55:18.238374 32190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:55:18.239750 32190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:55:18.262629 32199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:18.262539 32197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:18.262598 32196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:55:18.262966 32190 server_base.cc:1034] running on GCE node
I20260929 15:55:18.264248 32190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:55:18.265893 32190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:55:18.267158 32190 hybrid_clock.cc:648] HybridClock initialized: now 1790697318267085 us; error 44 us; skew 500 ppm
I20260929 15:55:18.267587 32190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:55:18 dist-test-slave-c51v krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697318, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:55:18.277762 32190 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:55:18.279670 32190 webserver.cc:460] Webserver started at http://127.15.196.254:40719/ using document root <none> and password file <none>
I20260929 15:55:18.280510 32190 fs_manager.cc:359] Metadata directory not provided
I20260929 15:55:18.280663 32190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:55:18.286855 32190 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 15:55:18.289992 32206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:18.291473 32190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 15:55:18.291659 32190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "7b34a40457ba4b21b923f0c90adbde22"
format_stamp: "Formatted at 2026-09-29 15:54:55 on dist-test-slave-c51v"
server_key: "885f25c31a1d8006566a3b3e1d3ede06"
server_key_iv: "4198fa19c6b775777ebeef8545f47782"
server_key_version: "encryptionkey@0"
I20260929 15:55:18.292742 32190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:55:18.329996 32190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:55:18.334299 32190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:55:18.334602 32190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:55:18.353808 32190 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:38147
I20260929 15:55:18.353842 32257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:38147 every 8 connection(s)
I20260929 15:55:18.355513 32190 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:55:18.357041 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 32190
I20260929 15:55:18.358079 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:36875
--local_ip_for_outbound_sockets=127.15.196.193
--tserver_master_addrs=127.15.196.254:38147
--webserver_port=45069
--webserver_interface=127.15.196.193
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:55:18.365921 32258 sys_catalog.cc:263] Verifying existing consensus state
I20260929 15:55:18.369712 32258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Bootstrap starting.
I20260929 15:55:18.390177 32258 log.cc:824] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Log is configured to *not* fsync() on all Append() calls
I20260929 15:55:18.402599 32258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 15:55:18.403169 32258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Bootstrap complete.
I20260929 15:55:18.412580 32258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } }
I20260929 15:55:18.413504 32258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b34a40457ba4b21b923f0c90adbde22, State: Initialized, Role: FOLLOWER
I20260929 15:55:18.414250 32258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } }
I20260929 15:55:18.414496 32258 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:55:18.414616 32258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:55:18.414800 32258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 1 FOLLOWER]: Advancing to term 2
I20260929 15:55:18.418848 32258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } }
I20260929 15:55:18.419445 32258 leader_election.cc:302] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b34a40457ba4b21b923f0c90adbde22; no voters:
I20260929 15:55:18.420136 32258 leader_election.cc:288] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 15:55:18.420292 32262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 15:55:18.421515 32262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 2 LEADER]: Becoming Leader. State: Replica: 7b34a40457ba4b21b923f0c90adbde22, State: Running, Role: LEADER
I20260929 15:55:18.422195 32262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } }
I20260929 15:55:18.422544 32258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:55:18.424861 32263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7b34a40457ba4b21b923f0c90adbde22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } } }
I20260929 15:55:18.425339 32263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [sys.catalog]: This master's current role is: LEADER
I20260929 15:55:18.425431 32264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b34a40457ba4b21b923f0c90adbde22. Latest consensus state: current_term: 2 leader_uuid: "7b34a40457ba4b21b923f0c90adbde22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } } }
I20260929 15:55:18.425782 32264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [sys.catalog]: This master's current role is: LEADER
I20260929 15:55:18.429901 32268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:55:18.439000 32268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:55:18.440539 32268 catalog_manager.cc:1259] Loaded cluster ID: a3f0216d378c4c01b9d602b907a2d0fa
I20260929 15:55:18.440719 32268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:55:18.444854 32268 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:55:18.446831 32268 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Loaded TSK: 0
I20260929 15:55:18.467501 32279 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 15:55:18.489398 32268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Generated new TSK 1
I20260929 15:55:18.489846 32268 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 15:55:18.719600 32260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:55:18.719892 32260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:55:18.720047 32260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:55:18.727708 32260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:55:18.727826 32260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:55:18.728001 32260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:55:18.738863 32260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:36875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=45069
--tserver_master_addrs=127.15.196.254:38147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:55:18.740305 32260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:55:18.741668 32260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:55:18.757099 32285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:18.757076 32286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:18.759562 32288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:55:18.764909 32260 server_base.cc:1034] running on GCE node
I20260929 15:55:18.765583 32260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:55:18.766753 32260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:55:18.767976 32260 hybrid_clock.cc:648] HybridClock initialized: now 1790697318767932 us; error 42 us; skew 500 ppm
I20260929 15:55:18.768314 32260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:55:18 dist-test-slave-c51v krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697318, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:55:18.771785 32260 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:55:18.773567 32260 webserver.cc:460] Webserver started at http://127.15.196.193:45069/ using document root <none> and password file <none>
I20260929 15:55:18.774330 32260 fs_manager.cc:359] Metadata directory not provided
I20260929 15:55:18.774492 32260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:55:18.780308 32260 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 15:55:18.783380 32295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:18.784871 32260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 15:55:18.785056 32260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "cbc9bc4e27e340ae9b340a346f6041d3"
format_stamp: "Formatted at 2026-09-29 15:54:55 on dist-test-slave-c51v"
server_key: "12e760a0d248384539a4ea62e2ad6578"
server_key_iv: "58974ee33090f3c5cea8c7f940cb823d"
server_key_version: "encryptionkey@0"
I20260929 15:55:18.786059 32260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:55:18.803107 32260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:55:18.807559 32260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:55:18.807852 32260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:55:18.808868 32260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:55:18.810372 32260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:55:18.810490 32260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:18.810617 32260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:55:18.810707 32260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:18.847636 32260 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:36875
I20260929 15:55:18.847728 32407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:36875 every 8 connection(s)
I20260929 15:55:18.849124 32260 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:55:18.855243 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 32260
I20260929 15:55:18.855922 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:33615
--local_ip_for_outbound_sockets=127.15.196.194
--tserver_master_addrs=127.15.196.254:38147
--webserver_port=46195
--webserver_interface=127.15.196.194
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 15:55:18 dist-test-slave-c51v krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697318, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:55:18.875083 32410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:18.852060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:53719 (local address 127.15.196.254:38147)
0929 15:55:18.852767 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:18.852787 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:18.853300 (+ 513us) server_negotiation.cc:407] Connection header received
0929 15:55:18.855553 (+ 2253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:18.855567 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:18.855911 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:18.856267 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:55:18.858105 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:18.860131 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:18.861724 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:18.862847 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:18.863060 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:18.867650 (+ 4590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:55:18.867659 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:55:18.867674 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:55:18.867714 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:55:18.870581 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:18.871594 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:18.871600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:18.871606 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:18.871765 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:18.872259 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:18.872262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:18.872264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:18.872715 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:55:18.872867 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:18.873790 (+ 923us) server_negotiation.cc:299] Negotiation successful
0929 15:55:18.874114 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":298,"threads_started":1}
I20260929 15:55:18.875681 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:18.852332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:53719)
0929 15:55:18.853113 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:18.853156 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 15:55:18.854913 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:55:18.856818 (+ 1905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:55:18.856825 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:55:18.857325 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:55:18.857868 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:18.857884 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:18.860315 (+ 2431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:18.860319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:18.861567 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:18.861580 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:18.863063 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:18.863067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:18.863404 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:18.864203 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:55:18.864237 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:55:18.867356 (+ 3119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:55:18.870757 (+ 3401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:55:18.870769 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:55:18.870789 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:55:18.871466 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:55:18.871885 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:55:18.871892 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:55:18.871899 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:55:18.872154 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:55:18.872860 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:55:18.872871 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:55:18.873312 (+ 441us) client_negotiation.cc:769] Sending connection context
0929 15:55:18.873631 (+ 319us) client_negotiation.cc:240] Negotiation successful
0929 15:55:18.874279 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":304,"threads_started":1}
I20260929 15:55:18.878652 32408 heartbeater.cc:347] Connected to a master server at 127.15.196.254:38147
I20260929 15:55:18.878970 32408 heartbeater.cc:464] Registering TS with master...
I20260929 15:55:18.879824 32408 heartbeater.cc:511] Master 127.15.196.254:38147 requested a full tablet report, sending...
I20260929 15:55:18.882586 32223 ts_manager.cc:194] Registered new tserver with Master: cbc9bc4e27e340ae9b340a346f6041d3 (127.15.196.193:36875)
I20260929 15:55:18.884222 32223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:53719
W20260929 15:55:19.204563 32412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:55:19.204849 32412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:55:19.204944 32412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:55:19.212162 32412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:55:19.212265 32412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:55:19.212376 32412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:55:19.222066 32412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:33615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=46195
--tserver_master_addrs=127.15.196.254:38147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:55:19.223464 32412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:55:19.224812 32412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:55:19.238096 32417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:19.238050 32418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:19.238166 32420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:55:19.238937 32412 server_base.cc:1034] running on GCE node
I20260929 15:55:19.239610 32412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:55:19.240558 32412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:55:19.241765 32412 hybrid_clock.cc:648] HybridClock initialized: now 1790697319241718 us; error 38 us; skew 500 ppm
I20260929 15:55:19.242095 32412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:55:19 dist-test-slave-c51v krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697319, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:55:19.245165 32412 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:55:19.246827 32412 webserver.cc:460] Webserver started at http://127.15.196.194:46195/ using document root <none> and password file <none>
I20260929 15:55:19.247612 32412 fs_manager.cc:359] Metadata directory not provided
I20260929 15:55:19.247774 32412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:55:19.253098 32412 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 15:55:19.255823 32427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:19.257153 32412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 15:55:19.257342 32412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "fa4812e4098c494994d66a14d5da2638"
format_stamp: "Formatted at 2026-09-29 15:54:56 on dist-test-slave-c51v"
server_key: "9733bfb2c26afbd45e740241bd6bfc64"
server_key_iv: "03990abe4eca8d470c210b6f2422b5f3"
server_key_version: "encryptionkey@0"
I20260929 15:55:19.258347 32412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:55:19.277537 32412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:55:19.281852 32412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:55:19.282110 32412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:55:19.283128 32412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:55:19.284461 32412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:55:19.284592 32412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:19.284727 32412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:55:19.284819 32412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:19.317848 32412 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:33615
I20260929 15:55:19.317901 32539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:33615 every 8 connection(s)
I20260929 15:55:19.319331 32412 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:55:19.323661 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 32412
I20260929 15:55:19.324476 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:38143
--local_ip_for_outbound_sockets=127.15.196.195
--tserver_master_addrs=127.15.196.254:38147
--webserver_port=42309
--webserver_interface=127.15.196.195
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 15:55:19 dist-test-slave-c51v krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697319, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:55:19.341845 32410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:19.321800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:33103 (local address 127.15.196.254:38147)
0929 15:55:19.321981 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:19.321985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:19.323246 (+ 1261us) server_negotiation.cc:407] Connection header received
0929 15:55:19.326109 (+ 2863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:19.326113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:19.326193 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:19.326279 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:55:19.328440 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.329117 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.331727 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.332212 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.332324 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:19.336178 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:55:19.336183 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:55:19.336187 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:55:19.336220 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:55:19.338771 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:19.339570 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:19.339575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:19.339579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:19.339704 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:19.340141 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:19.340146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:19.340150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:19.340423 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:55:19.340585 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:19.341266 (+ 681us) server_negotiation.cc:299] Negotiation successful
0929 15:55:19.341498 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 15:55:19.342936 32542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:19.322195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:33103)
0929 15:55:19.323053 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:19.323097 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 15:55:19.325790 (+ 2693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:55:19.326567 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:55:19.326581 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:55:19.327395 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:55:19.328221 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:19.328280 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.329300 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.329308 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:19.331551 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:19.331568 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.332442 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.332451 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:19.332779 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:19.333479 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:55:19.333503 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:55:19.335924 (+ 2421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:55:19.338911 (+ 2987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:55:19.338920 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:55:19.338937 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:55:19.339446 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:55:19.339825 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:55:19.339830 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:55:19.339834 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:55:19.340020 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:55:19.340560 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:55:19.340568 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:55:19.340916 (+ 348us) client_negotiation.cc:769] Sending connection context
0929 15:55:19.341179 (+ 263us) client_negotiation.cc:240] Negotiation successful
0929 15:55:19.341729 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":423,"threads_started":1}
I20260929 15:55:19.344662 32540 heartbeater.cc:347] Connected to a master server at 127.15.196.254:38147
I20260929 15:55:19.344995 32540 heartbeater.cc:464] Registering TS with master...
I20260929 15:55:19.345727 32540 heartbeater.cc:511] Master 127.15.196.254:38147 requested a full tablet report, sending...
I20260929 15:55:19.347206 32223 ts_manager.cc:194] Registered new tserver with Master: fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615)
I20260929 15:55:19.348057 32223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:33103
W20260929 15:55:19.665107 32543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:55:19.665465 32543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:55:19.665633 32543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:55:19.672912 32543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:55:19.673080 32543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:55:19.673256 32543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:55:19.683812 32543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:38143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=42309
--tserver_master_addrs=127.15.196.254:38147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:55:19.685220 32543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:55:19.686535 32543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:55:19.700793 32548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:19.700862 32551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:19.700891 32549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:55:19.702117 32543 server_base.cc:1034] running on GCE node
I20260929 15:55:19.702816 32543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:55:19.703913 32543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:55:19.705168 32543 hybrid_clock.cc:648] HybridClock initialized: now 1790697319705119 us; error 41 us; skew 500 ppm
I20260929 15:55:19.705680 32543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:55:19 dist-test-slave-c51v krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697319, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:55:19.709681 32543 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:55:19.711436 32543 webserver.cc:460] Webserver started at http://127.15.196.195:42309/ using document root <none> and password file <none>
I20260929 15:55:19.712245 32543 fs_manager.cc:359] Metadata directory not provided
I20260929 15:55:19.712446 32543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:55:19.718592 32543 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 15:55:19.721777 32558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:19.723508 32543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:55:19.723747 32543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "f84e04a0d55747e2a5d2a6bb4bc77147"
format_stamp: "Formatted at 2026-09-29 15:54:56 on dist-test-slave-c51v"
server_key: "809f49e30db4d3833b1e0bc29b21a61d"
server_key_iv: "948fcfe13082a92522cf93e481e08532"
server_key_version: "encryptionkey@0"
I20260929 15:55:19.724861 32543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:55:19.743976 32543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:55:19.749060 32543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:55:19.749393 32543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:55:19.750543 32543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:55:19.752162 32543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:55:19.752323 32543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:19.752480 32543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:55:19.752602 32543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:19.789399 32543 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:38143
I20260929 15:55:19.789443 32670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:38143 every 8 connection(s)
I20260929 15:55:19.790974 32543 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:55:19.791466 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 32543
Sep 29 15:55:19 dist-test-slave-c51v krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697319, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:55:19.808204 32410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:19.792665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36928 (local address 127.15.196.254:38147)
0929 15:55:19.792918 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:19.792928 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:19.793344 (+ 416us) server_negotiation.cc:407] Connection header received
0929 15:55:19.793556 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:19.793561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:19.793628 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:19.793740 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:55:19.795607 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.796625 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.798347 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.798802 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.798930 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:19.801934 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:55:19.801940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:55:19.801949 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:55:19.801979 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:55:19.805413 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:19.806159 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:19.806165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:19.806169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:19.806295 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:19.806679 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:19.806688 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:19.806691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:19.806969 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:55:19.807373 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:19.807566 (+ 193us) server_negotiation.cc:299] Negotiation successful
0929 15:55:19.807837 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 15:55:19.812912 32674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:19.794001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:35521 (local address 127.15.196.254:38147)
0929 15:55:19.794459 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:19.794465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:19.795631 (+ 1166us) server_negotiation.cc:407] Connection header received
0929 15:55:19.797729 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:19.797736 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:19.797805 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:19.797910 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:55:19.799367 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.800456 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.802030 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.802543 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.802693 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:19.806518 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:55:19.806522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:55:19.806525 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:55:19.806553 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:55:19.809458 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:19.810228 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:19.810233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:19.810236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:19.810365 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:19.810881 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:19.810887 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:19.810890 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:19.811157 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:55:19.811320 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:19.811964 (+ 644us) server_negotiation.cc:299] Negotiation successful
0929 15:55:19.812391 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":236,"threads_started":1}
I20260929 15:55:19.813783 32675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:19.794652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:35521)
0929 15:55:19.795494 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:19.795538 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 15:55:19.797492 (+ 1954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:55:19.798125 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:55:19.798133 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:55:19.798621 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:55:19.799186 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:19.799203 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.800641 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.800645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:19.801859 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:19.801871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.802815 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.802827 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:19.803156 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:19.803763 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:55:19.803795 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:55:19.806299 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:55:19.809642 (+ 3343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:55:19.809652 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:55:19.809668 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:55:19.810100 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:55:19.810501 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:55:19.810509 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:55:19.810514 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:55:19.810764 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:55:19.811314 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:55:19.811322 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:55:19.811644 (+ 322us) client_negotiation.cc:769] Sending connection context
0929 15:55:19.811903 (+ 259us) client_negotiation.cc:240] Negotiation successful
0929 15:55:19.812646 (+ 743us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":359,"threads_started":1}
I20260929 15:55:19.815032 32671 heartbeater.cc:347] Connected to a master server at 127.15.196.254:38147
I20260929 15:55:19.815397 32671 heartbeater.cc:464] Registering TS with master...
I20260929 15:55:19.816109 32671 heartbeater.cc:511] Master 127.15.196.254:38147 requested a full tablet report, sending...
I20260929 15:55:19.817425 32223 ts_manager.cc:194] Registered new tserver with Master: f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143)
I20260929 15:55:19.818221 32223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:35521
I20260929 15:55:19.823136 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:55:19.830463 32674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:19.824186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36936 (local address 127.15.196.254:38147)
0929 15:55:19.824412 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:19.824419 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:19.824750 (+ 331us) server_negotiation.cc:407] Connection header received
0929 15:55:19.824913 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:19.824917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:19.825080 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:19.825208 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:55:19.825630 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.826475 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.828110 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.828605 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.828721 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:19.829388 (+ 667us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:55:19.829686 (+ 298us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0929 15:55:19.830059 (+ 373us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36936: token expired
Metrics: {"server-negotiator.queue_time_us":65}
W20260929 15:55:19.830614 32674 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36936: token expired
W20260929 15:55:19.831113 32143 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.196.254:38147)
I20260929 15:55:19.845928 32674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:19.831772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36942 (local address 127.15.196.254:38147)
0929 15:55:19.832047 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:19.832054 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:19.832074 (+ 20us) server_negotiation.cc:407] Connection header received
0929 15:55:19.832228 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:19.832235 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:19.832307 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:19.832438 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:55:19.834210 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.835025 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.836670 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.837158 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.837323 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:19.840590 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:55:19.840596 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:55:19.840600 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:55:19.840635 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:55:19.843388 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:19.844091 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:19.844095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:19.844097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:19.844229 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:19.844604 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:19.844607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:19.844609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:19.844919 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:55:19.845128 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:19.845394 (+ 266us) server_negotiation.cc:299] Negotiation successful
0929 15:55:19.845569 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260929 15:55:19.849931 32223 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:55:19.858233 32223 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:55:19.887639 32408 heartbeater.cc:503] Master 127.15.196.254:38147 was elected leader, sending a full tablet report...
I20260929 15:55:19.894461 32683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:19.870667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:38143 (local address 127.0.0.1:46892)
0929 15:55:19.881242 (+ 10575us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:19.881269 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 15:55:19.881480 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:55:19.882773 (+ 1293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:55:19.882786 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:55:19.882844 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:55:19.883575 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:19.883594 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.886981 (+ 3387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.886991 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:19.891126 (+ 4135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:19.891141 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.893321 (+ 2180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.893331 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:19.893558 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:19.893625 (+ 67us) client_negotiation.cc:769] Sending connection context
0929 15:55:19.893753 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 15:55:19.893980 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10347,"mutex_wait_us":90,"thread_start_us":10165,"threads_started":1}
I20260929 15:55:19.894634 32685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:19.872038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46892 (local address 127.15.196.195:38143)
0929 15:55:19.880686 (+ 8648us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:19.880695 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:19.881798 (+ 1103us) server_negotiation.cc:407] Connection header received
0929 15:55:19.881945 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:19.881959 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:19.882436 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:19.883882 (+ 1446us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:55:19.884276 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.886733 (+ 2457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.891330 (+ 4597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.893053 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.893265 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:19.893477 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:19.894052 (+ 575us) server_negotiation.cc:299] Negotiation successful
0929 15:55:19.894254 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8454,"thread_start_us":8307,"threads_started":1}
I20260929 15:55:19.895501 32684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:19.870508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:36875 (local address 127.0.0.1:60804)
0929 15:55:19.879832 (+ 9324us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:19.879889 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 15:55:19.881823 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:55:19.886263 (+ 4440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:55:19.886268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:55:19.886295 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:55:19.886669 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:19.886686 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.888706 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.888711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:19.891905 (+ 3194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:19.891919 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.893811 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.893816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:19.894819 (+ 1003us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:19.894854 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 15:55:19.894973 (+ 119us) client_negotiation.cc:240] Negotiation successful
0929 15:55:19.895102 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8952,"mutex_wait_us":821,"thread_start_us":17896,"threads_started":1}
I20260929 15:55:19.895545 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:19.870695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60804 (local address 127.15.196.193:36875)
0929 15:55:19.885492 (+ 14797us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:19.885506 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:19.885545 (+ 39us) server_negotiation.cc:407] Connection header received
0929 15:55:19.885689 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:19.885701 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:19.886034 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:19.886285 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:55:19.886920 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.888539 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.892089 (+ 3550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.893255 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.893442 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:19.893618 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:19.895022 (+ 1404us) server_negotiation.cc:299] Negotiation successful
0929 15:55:19.895196 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14552,"thread_start_us":15762,"threads_started":1}
I20260929 15:55:19.898602 32686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:19.872824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46516 (local address 127.15.196.194:33615)
0929 15:55:19.889111 (+ 16287us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:19.889121 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:19.889154 (+ 33us) server_negotiation.cc:407] Connection header received
0929 15:55:19.889262 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:19.889270 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:19.889593 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:19.889855 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:55:19.890641 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.892295 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.895253 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.897049 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.897352 (+ 303us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:19.897542 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:19.898104 (+ 562us) server_negotiation.cc:299] Negotiation successful
0929 15:55:19.898295 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16084,"thread_start_us":16369,"threads_started":1}
I20260929 15:55:19.898674 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:19.871391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:33615 (local address 127.0.0.1:46516)
0929 15:55:19.887104 (+ 15713us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:19.887123 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:55:19.888314 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:55:19.889840 (+ 1526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:55:19.889845 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:55:19.889879 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:55:19.890391 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:19.890401 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.892487 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.892492 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:19.895089 (+ 2597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:19.895102 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:19.897714 (+ 2612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:19.897721 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:19.897901 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:19.897935 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 15:55:19.898032 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 15:55:19.898170 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15560,"mutex_wait_us":1,"thread_start_us":7839,"threads_started":1}
I20260929 15:55:19.899643 32606 tablet_service.cc:1467] Processing CreateTablet for tablet 25d0435b8f2e423490688796a892257e (DEFAULT_TABLE table=test-table [id=35ad30ba810540d2a961d9d8345a9eaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:55:19.899636 32343 tablet_service.cc:1467] Processing CreateTablet for tablet 25d0435b8f2e423490688796a892257e (DEFAULT_TABLE table=test-table [id=35ad30ba810540d2a961d9d8345a9eaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:55:19.901079 32343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25d0435b8f2e423490688796a892257e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:55:19.901237 32606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25d0435b8f2e423490688796a892257e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:55:19.903326 32475 tablet_service.cc:1467] Processing CreateTablet for tablet 25d0435b8f2e423490688796a892257e (DEFAULT_TABLE table=test-table [id=35ad30ba810540d2a961d9d8345a9eaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:55:19.904781 32475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25d0435b8f2e423490688796a892257e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:55:19.914177 32687 tablet_bootstrap.cc:492] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3: Bootstrap starting.
I20260929 15:55:19.917290 32687 tablet_bootstrap.cc:654] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3: Neither blocks nor log segments found. Creating new log.
I20260929 15:55:19.918190 32688 tablet_bootstrap.cc:492] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147: Bootstrap starting.
I20260929 15:55:19.918642 32687 log.cc:824] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3: Log is configured to *not* fsync() on all Append() calls
I20260929 15:55:19.921622 32688 tablet_bootstrap.cc:654] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147: Neither blocks nor log segments found. Creating new log.
I20260929 15:55:19.922349 32687 tablet_bootstrap.cc:492] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3: No bootstrap required, opened a new log
I20260929 15:55:19.922664 32687 ts_tablet_manager.cc:1397] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260929 15:55:19.922890 32690 tablet_bootstrap.cc:492] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638: Bootstrap starting.
I20260929 15:55:19.923161 32688 log.cc:824] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147: Log is configured to *not* fsync() on all Append() calls
I20260929 15:55:19.926337 32690 tablet_bootstrap.cc:654] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638: Neither blocks nor log segments found. Creating new log.
I20260929 15:55:19.926574 32688 tablet_bootstrap.cc:492] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147: No bootstrap required, opened a new log
I20260929 15:55:19.926856 32688 ts_tablet_manager.cc:1397] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260929 15:55:19.927572 32690 log.cc:824] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638: Log is configured to *not* fsync() on all Append() calls
I20260929 15:55:19.930757 32690 tablet_bootstrap.cc:492] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638: No bootstrap required, opened a new log
I20260929 15:55:19.931015 32690 ts_tablet_manager.cc:1397] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260929 15:55:19.932528 32687 raft_consensus.cc:348] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } }
I20260929 15:55:19.933004 32687 raft_consensus.cc:374] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:55:19.933128 32687 raft_consensus.cc:729] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbc9bc4e27e340ae9b340a346f6041d3, State: Initialized, Role: FOLLOWER
I20260929 15:55:19.933871 32687 consensus_queue.cc:260] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } }
I20260929 15:55:19.936580 32687 ts_tablet_manager.cc:1428] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3: Time spent starting tablet: real 0.014s user 0.009s sys 0.007s
I20260929 15:55:19.938552 32688 raft_consensus.cc:348] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } }
I20260929 15:55:19.939092 32688 raft_consensus.cc:374] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:55:19.939211 32688 raft_consensus.cc:729] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f84e04a0d55747e2a5d2a6bb4bc77147, State: Initialized, Role: FOLLOWER
I20260929 15:55:19.939873 32688 consensus_queue.cc:260] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } }
I20260929 15:55:19.940447 32690 raft_consensus.cc:348] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } }
I20260929 15:55:19.941074 32671 heartbeater.cc:503] Master 127.15.196.254:38147 was elected leader, sending a full tablet report...
I20260929 15:55:19.941740 32688 ts_tablet_manager.cc:1428] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260929 15:55:19.940878 32690 raft_consensus.cc:374] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:55:19.942163 32690 raft_consensus.cc:729] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4812e4098c494994d66a14d5da2638, State: Initialized, Role: FOLLOWER
I20260929 15:55:19.943517 32690 consensus_queue.cc:260] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } }
I20260929 15:55:19.944613 32540 heartbeater.cc:503] Master 127.15.196.254:38147 was elected leader, sending a full tablet report...
I20260929 15:55:19.945236 32690 ts_tablet_manager.cc:1428] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
I20260929 15:55:20.011328 32693 raft_consensus.cc:482] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:55:20.011650 32693 raft_consensus.cc:504] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } }
I20260929 15:55:20.013365 32693 leader_election.cc:288] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143), fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615)
I20260929 15:55:20.020218 32686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:20.014065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:60659 (local address 127.15.196.194:33615)
0929 15:55:20.014323 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:20.014332 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:20.014943 (+ 611us) server_negotiation.cc:407] Connection header received
0929 15:55:20.015145 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:20.015149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:20.015211 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:20.015333 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:55:20.015918 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:20.016610 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:20.018467 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:20.019226 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:20.019488 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:20.019565 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:20.019835 (+ 270us) server_negotiation.cc:299] Negotiation successful
0929 15:55:20.019961 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 15:55:20.020215 32685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:20.013811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:55321 (local address 127.15.196.195:38143)
0929 15:55:20.014028 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:20.014037 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:20.014712 (+ 675us) server_negotiation.cc:407] Connection header received
0929 15:55:20.014958 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:20.014962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:20.015025 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:20.015128 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:55:20.015750 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:20.016487 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:20.018461 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:20.019251 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:20.019457 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:20.019539 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:20.019817 (+ 278us) server_negotiation.cc:299] Negotiation successful
0929 15:55:20.019943 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 15:55:20.020398 32696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:20.013769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:38143 (local address 127.15.196.193:55321)
0929 15:55:20.014575 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:20.014593 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:55:20.014792 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:55:20.015146 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:55:20.015150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:55:20.015174 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:55:20.015583 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:20.015593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:20.016633 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:20.016639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:20.018306 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:20.018318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:20.019483 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:20.019489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:20.019649 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:20.019680 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 15:55:20.019792 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 15:55:20.019918 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":82,"thread_start_us":441,"threads_started":1}
I20260929 15:55:20.020757 32697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:20.013879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:33615 (local address 127.15.196.193:60659)
0929 15:55:20.014818 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:20.014835 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:55:20.014990 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:55:20.015353 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:55:20.015357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:55:20.015385 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:55:20.015757 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:20.015765 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:20.016788 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:20.016795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:20.018302 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:20.018314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:20.019525 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:20.019531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:20.019680 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:20.019706 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 15:55:20.019810 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 15:55:20.019931 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"mutex_wait_us":3,"thread_start_us":616,"threads_started":1}
I20260929 15:55:20.021422 32626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25d0435b8f2e423490688796a892257e" candidate_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" is_pre_election: true
I20260929 15:55:20.021631 32495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25d0435b8f2e423490688796a892257e" candidate_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4812e4098c494994d66a14d5da2638" is_pre_election: true
I20260929 15:55:20.021941 32626 raft_consensus.cc:2454] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbc9bc4e27e340ae9b340a346f6041d3 in term 0.
I20260929 15:55:20.022161 32495 raft_consensus.cc:2454] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbc9bc4e27e340ae9b340a346f6041d3 in term 0.
I20260929 15:55:20.022715 32298 leader_election.cc:302] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbc9bc4e27e340ae9b340a346f6041d3, f84e04a0d55747e2a5d2a6bb4bc77147; no voters:
I20260929 15:55:20.023138 32693 raft_consensus.cc:2781] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:55:20.023298 32693 raft_consensus.cc:482] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:55:20.023437 32693 raft_consensus.cc:3037] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:55:20.025636 32693 raft_consensus.cc:504] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } }
I20260929 15:55:20.026422 32693 leader_election.cc:288] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 1 election: Requested vote from peers f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143), fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615)
I20260929 15:55:20.026798 32626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25d0435b8f2e423490688796a892257e" candidate_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147"
I20260929 15:55:20.026994 32626 raft_consensus.cc:3037] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:55:20.026976 32495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25d0435b8f2e423490688796a892257e" candidate_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4812e4098c494994d66a14d5da2638"
I20260929 15:55:20.027182 32495 raft_consensus.cc:3037] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:55:20.029434 32626 raft_consensus.cc:2454] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbc9bc4e27e340ae9b340a346f6041d3 in term 1.
I20260929 15:55:20.029560 32495 raft_consensus.cc:2454] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbc9bc4e27e340ae9b340a346f6041d3 in term 1.
I20260929 15:55:20.029870 32298 leader_election.cc:302] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbc9bc4e27e340ae9b340a346f6041d3, f84e04a0d55747e2a5d2a6bb4bc77147; no voters:
I20260929 15:55:20.030134 32693 raft_consensus.cc:2781] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:55:20.030572 32693 raft_consensus.cc:686] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 LEADER]: Becoming Leader. State: Replica: cbc9bc4e27e340ae9b340a346f6041d3, State: Running, Role: LEADER
I20260929 15:55:20.031122 32693 consensus_queue.cc:237] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } }
I20260929 15:55:20.036387 32223 catalog_manager.cc:5495] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to cbc9bc4e27e340ae9b340a346f6041d3 (127.15.196.193). New cstate: current_term: 1 leader_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } health_report { overall_health: UNKNOWN } } }
W20260929 15:55:20.042774 32672 tablet.cc:2369] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 15:55:20.071668 32541 tablet.cc:2369] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:55:20.084563 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:20.079022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60816 (local address 127.15.196.193:36875)
0929 15:55:20.079228 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:20.079232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:20.079250 (+ 18us) server_negotiation.cc:407] Connection header received
0929 15:55:20.079534 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:20.079542 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:20.079670 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:20.079868 (+ 198us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:55:20.080188 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:20.081114 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:20.082539 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:20.083047 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:20.083148 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:20.083636 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:55:20.084129 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:55:20.084252 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:20.084297 (+ 45us) server_negotiation.cc:299] Negotiation successful
0929 15:55:20.084361 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 15:55:20.093102 32495 raft_consensus.cc:1261] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638 [term 1 FOLLOWER]: Refusing update from remote peer cbc9bc4e27e340ae9b340a346f6041d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:55:20.093103 32626 raft_consensus.cc:1261] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147 [term 1 FOLLOWER]: Refusing update from remote peer cbc9bc4e27e340ae9b340a346f6041d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:55:20.094229 32698 consensus_queue.cc:1035] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:55:20.094620 32693 consensus_queue.cc:1035] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 15:55:20.101425 32409 tablet.cc:2369] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:55:20.107030 32700 mvcc.cc:205] Tried to move back new op lower bound from 7334696223090024448 to 7334696222851952640. Current Snapshot: MvccSnapshot[applied={T|T < 7334696223090024448}]
I20260929 15:55:20.108556 32701 mvcc.cc:205] Tried to move back new op lower bound from 7334696223090024448 to 7334696222851952640. Current Snapshot: MvccSnapshot[applied={T|T < 7334696223090024448}]
I20260929 15:55:20.122728 32223 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:55:20.122994 32223 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:55:20.126696 32223 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Sending DeleteTablet for 3 replicas of tablet 25d0435b8f2e423490688796a892257e
I20260929 15:55:20.127940 32606 tablet_service.cc:1514] Processing DeleteTablet for tablet 25d0435b8f2e423490688796a892257e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:55:20 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:46892
I20260929 15:55:20.127987 32343 tablet_service.cc:1514] Processing DeleteTablet for tablet 25d0435b8f2e423490688796a892257e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:55:20 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:60804
I20260929 15:55:20.128852 32710 tablet_replica.cc:330] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147: stopping tablet replica
I20260929 15:55:20.129300 32475 tablet_service.cc:1514] Processing DeleteTablet for tablet 25d0435b8f2e423490688796a892257e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:55:20 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:46516
I20260929 15:55:20.129403 32710 raft_consensus.cc:2229] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:55:20.130026 32710 raft_consensus.cc:2258] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:55:20.130447 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 32260
I20260929 15:55:20.134359 32710 ts_tablet_manager.cc:1905] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:55:20.140307 32712 tablet_replica.cc:330] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638: stopping tablet replica
I20260929 15:55:20.140805 32712 raft_consensus.cc:2229] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:55:20.141360 32712 raft_consensus.cc:2258] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:55:20.143322 32710 ts_tablet_manager.cc:1918] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:55:20.143541 32710 log.cc:1196] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/25d0435b8f2e423490688796a892257e
I20260929 15:55:20.143831 32712 ts_tablet_manager.cc:1905] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:55:20.144266 32710 ts_tablet_manager.cc:1939] T 25d0435b8f2e423490688796a892257e P f84e04a0d55747e2a5d2a6bb4bc77147: Deleting consensus metadata
I20260929 15:55:20.146301 32209 catalog_manager.cc:4841] TS f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143): tablet 25d0435b8f2e423490688796a892257e (table test-table [id=35ad30ba810540d2a961d9d8345a9eaa]) successfully deleted
I20260929 15:55:20.150519 32712 ts_tablet_manager.cc:1918] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:55:20.150700 32712 log.cc:1196] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/25d0435b8f2e423490688796a892257e
I20260929 15:55:20.151230 32712 ts_tablet_manager.cc:1939] T 25d0435b8f2e423490688796a892257e P fa4812e4098c494994d66a14d5da2638: Deleting consensus metadata
I20260929 15:55:20.153147 32210 catalog_manager.cc:4841] TS fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615): tablet 25d0435b8f2e423490688796a892257e (table test-table [id=35ad30ba810540d2a961d9d8345a9eaa]) successfully deleted
I20260929 15:55:20.153874 32711 tablet_replica.cc:330] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3: stopping tablet replica
I20260929 15:55:20.154352 32711 raft_consensus.cc:2229] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:55:20.155074 32682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:20.131850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60830 (local address 127.15.196.193:36875)
0929 15:55:20.132056 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:20.132062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:20.132082 (+ 20us) server_negotiation.cc:407] Connection header received
0929 15:55:20.132163 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:20.132184 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:20.132288 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:20.132419 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:55:20.134867 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:20.136606 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:20.139253 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:20.140238 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:20.140391 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:20.145184 (+ 4793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:55:20.145206 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:55:20.145230 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:55:20.145296 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:55:20.150790 (+ 5494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:20.152015 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:20.152029 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:20.152033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:20.152256 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:20.152855 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:20.152864 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:20.152869 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:20.153468 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:55:20.153748 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:20.154353 (+ 605us) server_negotiation.cc:299] Negotiation successful
0929 15:55:20.154664 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 15:55:20.155040 32711 raft_consensus.cc:2258] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:55:20.155874 32403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:55:20.271754 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 32260
W20260929 15:55:20.290114 32207 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.196.193:36875 (error 108)
I20260929 15:55:20.290402 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 32412
I20260929 15:55:20.292263 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:20.291773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:36875 (local address 127.0.0.1:60844)
0929 15:55:20.291961 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:20.292074 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260929 15:55:20.292522 32207 catalog_manager.cc:4584] TS cbc9bc4e27e340ae9b340a346f6041d3 (127.15.196.193:36875): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 25d0435b8f2e423490688796a892257e: Network error: Client connection negotiation failed: client connection to 127.15.196.193:36875: connect: Connection refused (error 111)
I20260929 15:55:20.304368 32686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:20.291335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46520 (local address 127.15.196.194:33615)
0929 15:55:20.291503 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:20.291509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:20.291590 (+ 81us) server_negotiation.cc:407] Connection header received
0929 15:55:20.291789 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:20.291793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:20.291909 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:20.292013 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:55:20.293459 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:20.294179 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:20.295727 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:20.296124 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:20.296227 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:20.299046 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:55:20.299060 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:55:20.299075 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:55:20.299109 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:55:20.301783 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:20.302452 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:20.302459 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:20.302461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:20.302585 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:20.302907 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:20.302910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:20.302912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:20.303271 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:55:20.303577 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:20.303853 (+ 276us) server_negotiation.cc:299] Negotiation successful
0929 15:55:20.304016 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 15:55:20.304991 32535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:55:20.346516 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:20.345937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:36875 (local address 127.0.0.1:60850)
0929 15:55:20.346147 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:20.346304 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 15:55:20.347589 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:20.347226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:36875 (local address 127.0.0.1:60862)
0929 15:55:20.347392 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:20.347481 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 15:55:20.408236 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 32412
I20260929 15:55:20.408483 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:20.407786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:36875 (local address 127.0.0.1:60874)
0929 15:55:20.408208 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:20.408294 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265}
I20260929 15:55:20.410069 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:20.409730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:36875 (local address 127.0.0.1:60878)
0929 15:55:20.409882 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:20.409962 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 15:55:20.425781 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 32543
I20260929 15:55:20.439091 32685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:20.426485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46904 (local address 127.15.196.195:38143)
0929 15:55:20.426648 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:20.426653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:20.426677 (+ 24us) server_negotiation.cc:407] Connection header received
0929 15:55:20.426816 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:20.426820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:20.426883 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:20.426966 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:55:20.428406 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:20.429242 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:20.430583 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:20.431060 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:20.431194 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:20.433862 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:55:20.433877 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:55:20.433894 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:55:20.433931 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:55:20.436618 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:20.437257 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:20.437264 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:20.437267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:20.437405 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:20.437753 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:20.437757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:20.437760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:20.438123 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:55:20.438351 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:20.438649 (+ 298us) server_negotiation.cc:299] Negotiation successful
0929 15:55:20.438840 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 15:55:20.439940 32666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:55:20.490732 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:20.490244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:36875 (local address 127.0.0.1:60892)
0929 15:55:20.490461 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:20.490551 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260929 15:55:20.491858 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:20.491472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:36875 (local address 127.0.0.1:60908)
0929 15:55:20.491673 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:20.491746 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:36875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260929 15:55:20.530925 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 32543
I20260929 15:55:20.547732 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 32190
I20260929 15:55:20.548920 32253 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:55:20.663853 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 32190
I20260929 15:55:40.682444 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:38147
--webserver_interface=127.15.196.254
--webserver_port=40719
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:38147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:55:41.017139 32718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:55:41.017442 32718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:55:41.017603 32718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:55:41.025028 32718 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 15:55:41.025138 32718 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 15:55:41.025226 32718 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 15:55:41.025331 32718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:55:41.025425 32718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:55:41.025554 32718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:55:41.025640 32718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:55:41.025724 32718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:55:41.036139 32718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:38147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:38147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=40719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:55:41.037539 32718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:55:41.038976 32718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:55:41.051910 32723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:41.051873 32724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:41.051906 32726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:55:41.052129 32718 server_base.cc:1034] running on GCE node
I20260929 15:55:41.053534 32718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:55:41.054927 32718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:55:41.056146 32718 hybrid_clock.cc:648] HybridClock initialized: now 1790697341056086 us; error 32 us; skew 500 ppm
I20260929 15:55:41.056450 32718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:55:41 dist-test-slave-c51v krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697341, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:55:41.065011 32718 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:55:41.066571 32718 webserver.cc:460] Webserver started at http://127.15.196.254:40719/ using document root <none> and password file <none>
I20260929 15:55:41.067338 32718 fs_manager.cc:359] Metadata directory not provided
I20260929 15:55:41.067446 32718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:55:41.072962 32718 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 15:55:41.075871 32733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:41.077241 32718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:55:41.077389 32718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "7b34a40457ba4b21b923f0c90adbde22"
format_stamp: "Formatted at 2026-09-29 15:54:55 on dist-test-slave-c51v"
server_key: "885f25c31a1d8006566a3b3e1d3ede06"
server_key_iv: "4198fa19c6b775777ebeef8545f47782"
server_key_version: "encryptionkey@0"
I20260929 15:55:41.078327 32718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:55:41.095572 32718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:55:41.099766 32718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:55:41.100005 32718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:55:41.118909 32718 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:38147
I20260929 15:55:41.118937 316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:38147 every 8 connection(s)
I20260929 15:55:41.120476 32718 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:55:41.127063 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 32718
I20260929 15:55:41.127588 317 sys_catalog.cc:263] Verifying existing consensus state
I20260929 15:55:41.128050 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:36875
--local_ip_for_outbound_sockets=127.15.196.193
--tserver_master_addrs=127.15.196.254:38147
--webserver_port=45069
--webserver_interface=127.15.196.193
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:55:41.130957 317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Bootstrap starting.
I20260929 15:55:41.151711 317 log.cc:824] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Log is configured to *not* fsync() on all Append() calls
I20260929 15:55:41.168017 317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 15:55:41.168588 317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Bootstrap complete.
I20260929 15:55:41.177757 317 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } }
I20260929 15:55:41.178751 317 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b34a40457ba4b21b923f0c90adbde22, State: Initialized, Role: FOLLOWER
I20260929 15:55:41.179432 317 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } }
I20260929 15:55:41.179697 317 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:55:41.179829 317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:55:41.180009 317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 2 FOLLOWER]: Advancing to term 3
I20260929 15:55:41.184242 317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } }
I20260929 15:55:41.184756 317 leader_election.cc:302] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b34a40457ba4b21b923f0c90adbde22; no voters:
I20260929 15:55:41.185431 317 leader_election.cc:288] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 15:55:41.185568 321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 15:55:41.187971 317 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:55:41.189663 321 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [term 3 LEADER]: Becoming Leader. State: Replica: 7b34a40457ba4b21b923f0c90adbde22, State: Running, Role: LEADER
I20260929 15:55:41.190361 321 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } }
I20260929 15:55:41.193178 323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b34a40457ba4b21b923f0c90adbde22. Latest consensus state: current_term: 3 leader_uuid: "7b34a40457ba4b21b923f0c90adbde22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } } }
I20260929 15:55:41.193639 323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [sys.catalog]: This master's current role is: LEADER
I20260929 15:55:41.194157 322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7b34a40457ba4b21b923f0c90adbde22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b34a40457ba4b21b923f0c90adbde22" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38147 } } }
I20260929 15:55:41.194482 322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22 [sys.catalog]: This master's current role is: LEADER
I20260929 15:55:41.204634 330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:55:41.214622 330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:55:41.216547 330 catalog_manager.cc:1259] Loaded cluster ID: a3f0216d378c4c01b9d602b907a2d0fa
I20260929 15:55:41.216696 330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:55:41.221326 339 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 15:55:41.221850 330 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:55:41.223325 330 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Loaded TSK: 0
I20260929 15:55:41.223488 330 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b34a40457ba4b21b923f0c90adbde22: Loaded TSK: 1
F20260929 15:55:41.224334 330 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd64cd7d9ad google::LogMessage::Fail() at ??:0
@ 0x7fd64cd8088a google::LogMessage::SendToLog() at ??:0
@ 0x7fd64cd7d4de google::LogMessage::Flush() at ??:0
@ 0x7fd64cd7ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd650ae17d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fd650ad073e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fd6627ac1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fd6627ab783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fd6628372a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fd6628370c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fd6628815e9 std::function<>::operator()() at ??:0
@ 0x7fd6627afcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fd6627adc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd662835610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fd662835439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd64f9c9345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd64f9cc91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fd64f9cc799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd64f9a7eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd652c816db start_thread at ??:0
@ 0x7fd64bc1761f clone at ??:0
W20260929 15:55:41.512852 319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:55:41.513185 319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:55:41.513360 319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:55:41.521034 319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:55:41.521184 319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:55:41.521370 319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:55:41.531639 319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:36875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=45069
--tserver_master_addrs=127.15.196.254:38147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:55:41.533056 319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:55:41.534435 319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:55:41.548130 368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:41.548213 365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:41.548193 366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:55:41.549050 319 server_base.cc:1034] running on GCE node
I20260929 15:55:41.549780 319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:55:41.550812 319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:55:41.552047 319 hybrid_clock.cc:648] HybridClock initialized: now 1790697341552003 us; error 53 us; skew 500 ppm
I20260929 15:55:41.552379 319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:55:41 dist-test-slave-c51v krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697341, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:55:41.555701 319 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:55:41.557157 319 webserver.cc:460] Webserver started at http://127.15.196.193:45069/ using document root <none> and password file <none>
I20260929 15:55:41.557883 319 fs_manager.cc:359] Metadata directory not provided
I20260929 15:55:41.558032 319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:55:41.563702 319 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 15:55:41.566641 375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:41.568109 319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:55:41.568313 319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "cbc9bc4e27e340ae9b340a346f6041d3"
format_stamp: "Formatted at 2026-09-29 15:54:55 on dist-test-slave-c51v"
server_key: "12e760a0d248384539a4ea62e2ad6578"
server_key_iv: "58974ee33090f3c5cea8c7f940cb823d"
server_key_version: "encryptionkey@0"
I20260929 15:55:41.569300 319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:55:41.586644 319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:55:41.590773 319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:55:41.591059 319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:55:41.592178 319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:55:41.594038 382 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260929 15:55:41.600215 319 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 15:55:41.600378 319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260929 15:55:41.600519 319 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 15:55:41.603811 319 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 15:55:41.603921 319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260929 15:55:41.604311 382 tablet_bootstrap.cc:492] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3: Bootstrap starting.
I20260929 15:55:41.613348 382 log.cc:824] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3: Log is configured to *not* fsync() on all Append() calls
I20260929 15:55:41.628022 382 tablet_bootstrap.cc:492] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 15:55:41.628964 382 tablet_bootstrap.cc:492] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3: Bootstrap complete.
I20260929 15:55:41.629550 382 ts_tablet_manager.cc:1397] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3: Time spent bootstrapping tablet: real 0.026s user 0.022s sys 0.000s
I20260929 15:55:41.640444 382 raft_consensus.cc:348] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } }
I20260929 15:55:41.642031 382 raft_consensus.cc:729] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbc9bc4e27e340ae9b340a346f6041d3, State: Initialized, Role: FOLLOWER
I20260929 15:55:41.642977 382 consensus_queue.cc:260] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } }
I20260929 15:55:41.648830 382 ts_tablet_manager.cc:1428] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3: Time spent starting tablet: real 0.019s user 0.021s sys 0.000s
I20260929 15:55:41.651655 319 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:36875
I20260929 15:55:41.651782 490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:36875 every 8 connection(s)
I20260929 15:55:41.652918 319 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:55:41.653522 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 319
I20260929 15:55:41.654520 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:33615
--local_ip_for_outbound_sockets=127.15.196.194
--tserver_master_addrs=127.15.196.254:38147
--webserver_port=46195
--webserver_interface=127.15.196.194
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:55:41.658257 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:41.655925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:55333)
0929 15:55:41.656845 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:41.657123 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":347,"threads_started":1}
W20260929 15:55:41.658991 491 heartbeater.cc:644] Failed to heartbeat to 127.15.196.254:38147 (0 consecutive failures): Network error: Failed to ping master at 127.15.196.254:38147: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
I20260929 15:55:41.659978 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:41.659536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:36127)
0929 15:55:41.659735 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:41.659829 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 15:55:41.660928 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:41.660501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:36545)
0929 15:55:41.660671 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:41.660776 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 15:55:41.661165 491 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 15:55:41.983875 494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:55:41.984203 494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:55:41.984381 494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:55:41.991645 494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:55:41.991803 494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:55:41.991979 494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:55:42.001704 494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:33615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=46195
--tserver_master_addrs=127.15.196.254:38147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:55:42.003026 494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:55:42.004380 494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:55:42.017379 499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:42.017326 500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:42.017290 502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:55:42.018359 494 server_base.cc:1034] running on GCE node
I20260929 15:55:42.018961 494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:55:42.019944 494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:55:42.021165 494 hybrid_clock.cc:648] HybridClock initialized: now 1790697342021122 us; error 53 us; skew 500 ppm
I20260929 15:55:42.021485 494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:55:42 dist-test-slave-c51v krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697342, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:55:42.024588 494 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:55:42.025993 494 webserver.cc:460] Webserver started at http://127.15.196.194:46195/ using document root <none> and password file <none>
I20260929 15:55:42.026681 494 fs_manager.cc:359] Metadata directory not provided
I20260929 15:55:42.026829 494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:55:42.032166 494 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:55:42.034852 509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:42.036274 494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 15:55:42.036458 494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "fa4812e4098c494994d66a14d5da2638"
format_stamp: "Formatted at 2026-09-29 15:54:56 on dist-test-slave-c51v"
server_key: "9733bfb2c26afbd45e740241bd6bfc64"
server_key_iv: "03990abe4eca8d470c210b6f2422b5f3"
server_key_version: "encryptionkey@0"
I20260929 15:55:42.037459 494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:55:42.078281 494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:55:42.082644 494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:55:42.082942 494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:55:42.084030 494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:55:42.085443 494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:55:42.085553 494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:42.085687 494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:55:42.085775 494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:42.118443 494 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:33615
I20260929 15:55:42.118530 621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:33615 every 8 connection(s)
I20260929 15:55:42.120026 494 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:55:42.124814 624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:42.122957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:54375)
0929 15:55:42.123705 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:42.123947 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":307,"threads_started":1}
W20260929 15:55:42.125478 622 heartbeater.cc:644] Failed to heartbeat to 127.15.196.254:38147 (0 consecutive failures): Network error: Failed to ping master at 127.15.196.254:38147: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
I20260929 15:55:42.126296 624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:42.125983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:34389)
0929 15:55:42.126120 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:42.126187 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 15:55:42.127122 624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:42.126849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:56863)
0929 15:55:42.126958 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:42.127026 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 15:55:42.127420 622 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 15:55:42.130184 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 494
I20260929 15:55:42.130800 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:38143
--local_ip_for_outbound_sockets=127.15.196.195
--tserver_master_addrs=127.15.196.254:38147
--webserver_port=42309
--webserver_interface=127.15.196.195
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:39077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:55:42.460716 625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:55:42.460994 625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:55:42.461158 625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:55:42.468279 625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:55:42.468384 625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:55:42.468592 625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:55:42.478508 625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:39077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:38143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=42309
--tserver_master_addrs=127.15.196.254:38147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:55:42.479828 625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:55:42.481195 625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:55:42.494565 631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:42.494545 633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:42.494772 630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:55:42.495352 625 server_base.cc:1034] running on GCE node
I20260929 15:55:42.496028 625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:55:42.497022 625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:55:42.498270 625 hybrid_clock.cc:648] HybridClock initialized: now 1790697342498204 us; error 45 us; skew 500 ppm
I20260929 15:55:42.498627 625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:55:42 dist-test-slave-c51v krb5kdc[31637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697342, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:55:42.501915 625 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:55:42.503473 625 webserver.cc:460] Webserver started at http://127.15.196.195:42309/ using document root <none> and password file <none>
I20260929 15:55:42.504145 625 fs_manager.cc:359] Metadata directory not provided
I20260929 15:55:42.504251 625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:55:42.509583 625 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 15:55:42.512183 640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:42.513520 625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:55:42.513695 625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "f84e04a0d55747e2a5d2a6bb4bc77147"
format_stamp: "Formatted at 2026-09-29 15:54:56 on dist-test-slave-c51v"
server_key: "809f49e30db4d3833b1e0bc29b21a61d"
server_key_iv: "948fcfe13082a92522cf93e481e08532"
server_key_version: "encryptionkey@0"
I20260929 15:55:42.514642 625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:55:42.537745 625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:55:42.542641 625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:55:42.542944 625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:55:42.544045 625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:55:42.545423 625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:55:42.545531 625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:42.545665 625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:55:42.545756 625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:42.579522 625 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:38143
I20260929 15:55:42.579584 752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:38143 every 8 connection(s)
I20260929 15:55:42.581070 625 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:55:42.584141 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 625
I20260929 15:55:42.586458 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:42.584259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:55153)
0929 15:55:42.585153 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:42.585482 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":314,"threads_started":1}
W20260929 15:55:42.587077 753 heartbeater.cc:644] Failed to heartbeat to 127.15.196.254:38147 (0 consecutive failures): Network error: Failed to ping master at 127.15.196.254:38147: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
I20260929 15:55:42.587868 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:42.587585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:56359)
0929 15:55:42.587710 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:42.587774 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 15:55:42.588605 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:42.588327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:52627)
0929 15:55:42.588438 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:42.588504 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260929 15:55:42.588832 753 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 15:55:42.662361 757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:42.661643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:34827)
0929 15:55:42.662064 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:42.662151 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":208,"threads_started":1}
I20260929 15:55:43.128822 758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:43.127993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:56423)
0929 15:55:43.128475 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:43.128586 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":230,"threads_started":1}
I20260929 15:55:43.493937 759 raft_consensus.cc:482] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:55:43.494295 759 raft_consensus.cc:504] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } }
I20260929 15:55:43.496080 759 leader_election.cc:288] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143), fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615)
Sep 29 15:55:43 dist-test-slave-c51v krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697341, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
Sep 29 15:55:43 dist-test-slave-c51v krb5kdc[31637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697341, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:55:43.527243 763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:43.496878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:38887 (local address 127.15.196.194:33615)
0929 15:55:43.507097 (+ 10219us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:43.507116 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:43.507867 (+ 751us) server_negotiation.cc:407] Connection header received
0929 15:55:43.509881 (+ 2014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:43.509888 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:43.510192 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:43.510507 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:55:43.512592 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:43.513727 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:43.515541 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:43.516148 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:43.516349 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:43.520533 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:55:43.520540 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:55:43.520552 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:55:43.520609 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:55:43.523532 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:43.524368 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:43.524375 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:43.524381 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:43.524531 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:43.524979 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:43.524983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:43.524985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:43.525485 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:55:43.525686 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:43.526468 (+ 782us) server_negotiation.cc:299] Negotiation successful
0929 15:55:43.526827 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9988,"thread_start_us":9782,"threads_started":1}
I20260929 15:55:43.527310 761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:43.496453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:38143 (local address 127.15.196.193:37403)
0929 15:55:43.507525 (+ 11072us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:43.507564 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 15:55:43.508148 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:55:43.510623 (+ 2475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:55:43.510634 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:55:43.511620 (+ 986us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:55:43.512404 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:43.512424 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:43.513907 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:43.513911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:43.515398 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:43.515410 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:43.516405 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:43.516413 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:43.516594 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:43.517384 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:55:43.517410 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:55:43.520033 (+ 2623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:55:43.523677 (+ 3644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:55:43.523689 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:55:43.523704 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:55:43.524236 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:55:43.524658 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:55:43.524665 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:55:43.524669 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:55:43.524861 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:55:43.525625 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:55:43.525633 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:55:43.525960 (+ 327us) client_negotiation.cc:769] Sending connection context
0929 15:55:43.526239 (+ 279us) client_negotiation.cc:240] Negotiation successful
0929 15:55:43.526795 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10924,"mutex_wait_us":147,"spinlock_wait_cycles":47360,"thread_start_us":10966,"threads_started":1}
I20260929 15:55:43.527412 760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:43.496498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:33615 (local address 127.15.196.193:38887)
0929 15:55:43.507710 (+ 11212us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:43.507735 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 15:55:43.508166 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:55:43.510623 (+ 2457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:55:43.510634 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:55:43.511619 (+ 985us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:55:43.512404 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:43.512424 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:43.513899 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:43.513903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:43.515400 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:55:43.515412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:55:43.516354 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:55:43.516361 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:55:43.516593 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:43.517451 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:55:43.517467 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:55:43.520272 (+ 2805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:55:43.523677 (+ 3405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:55:43.523694 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:55:43.523704 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:55:43.524219 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:55:43.524648 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:55:43.524653 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:55:43.524658 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:55:43.524853 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:55:43.525624 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:55:43.525633 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:55:43.525961 (+ 328us) client_negotiation.cc:769] Sending connection context
0929 15:55:43.526239 (+ 278us) client_negotiation.cc:240] Negotiation successful
0929 15:55:43.526948 (+ 709us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11082,"mutex_wait_us":450,"spinlock_wait_cycles":24192,"thread_start_us":10576,"threads_started":1}
I20260929 15:55:43.527262 762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:43.496596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:37403 (local address 127.15.196.195:38143)
0929 15:55:43.506657 (+ 10061us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:43.506678 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:43.507745 (+ 1067us) server_negotiation.cc:407] Connection header received
0929 15:55:43.509788 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:43.509796 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:43.510184 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:43.510494 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:55:43.512597 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:43.513705 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:43.515546 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:43.516152 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:43.516361 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:43.520326 (+ 3965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:55:43.520334 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:55:43.520348 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:55:43.520387 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:55:43.523529 (+ 3142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:43.524380 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:43.524387 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:43.524392 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:43.524539 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:43.524983 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:43.524987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:43.524989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:43.525485 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:55:43.525684 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:43.526462 (+ 778us) server_negotiation.cc:299] Negotiation successful
0929 15:55:43.526831 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9859,"thread_start_us":9732,"threads_started":1}
W20260929 15:55:43.530395 378 leader_election.cc:341] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143): Not found: Tablet not found: 25d0435b8f2e423490688796a892257e
W20260929 15:55:43.530655 379 leader_election.cc:341] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615): Not found: Tablet not found: 25d0435b8f2e423490688796a892257e
I20260929 15:55:43.530788 379 leader_election.cc:302] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cbc9bc4e27e340ae9b340a346f6041d3; no voters: f84e04a0d55747e2a5d2a6bb4bc77147, fa4812e4098c494994d66a14d5da2638
I20260929 15:55:43.531164 759 raft_consensus.cc:2726] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 15:55:43.590309 764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:43.589416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:57065)
0929 15:55:43.589905 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:43.590059 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":222,"threads_started":1}
I20260929 15:55:43.663844 760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:43.663417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:56899)
0929 15:55:43.663592 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:43.663706 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 15:55:44.130613 765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:44.129835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:58893)
0929 15:55:44.130332 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:44.130422 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":272,"threads_started":1}
I20260929 15:55:44.592052 766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:44.591258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:44011)
0929 15:55:44.591802 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:44.591892 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":283,"threads_started":1}
I20260929 15:55:44.665480 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:44.664779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:55945)
0929 15:55:44.665221 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:44.665307 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":224,"threads_started":1}
I20260929 15:55:45.132294 768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:45.131576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:60667)
0929 15:55:45.132031 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:45.132111 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":224,"threads_started":1}
I20260929 15:55:45.593703 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:45.592952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:56959)
0929 15:55:45.593433 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:45.593521 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":250,"threads_started":1}
I20260929 15:55:45.600517 770 raft_consensus.cc:482] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:55:45.600726 770 raft_consensus.cc:504] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } }
I20260929 15:55:45.601410 770 leader_election.cc:288] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143), fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615)
W20260929 15:55:45.602319 378 leader_election.cc:341] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143): Not found: Tablet not found: 25d0435b8f2e423490688796a892257e
W20260929 15:55:45.602469 379 leader_election.cc:341] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615): Not found: Tablet not found: 25d0435b8f2e423490688796a892257e
I20260929 15:55:45.602675 379 leader_election.cc:302] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cbc9bc4e27e340ae9b340a346f6041d3; no voters: f84e04a0d55747e2a5d2a6bb4bc77147, fa4812e4098c494994d66a14d5da2638
I20260929 15:55:45.602941 770 raft_consensus.cc:2726] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 15:55:45.667229 771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:45.666422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:53925)
0929 15:55:45.666934 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:45.667050 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":279,"threads_started":1}
I20260929 15:55:46.133916 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:46.133160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:59509)
0929 15:55:46.133643 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:46.133731 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":249,"threads_started":1}
I20260929 15:55:46.595502 773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:46.594633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:53849)
0929 15:55:46.595171 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:46.595259 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":308,"threads_started":1}
I20260929 15:55:46.669015 774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:46.668284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:47737)
0929 15:55:46.668741 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:46.668835 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":215,"threads_started":1}
I20260929 15:55:47.135511 775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:47.134780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:44599)
0929 15:55:47.135201 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:47.135335 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":206,"threads_started":1}
I20260929 15:55:47.204653 776 raft_consensus.cc:482] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:55:47.204919 776 raft_consensus.cc:504] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } }
I20260929 15:55:47.205582 776 leader_election.cc:288] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143), fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615)
W20260929 15:55:47.206550 378 leader_election.cc:341] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143): Not found: Tablet not found: 25d0435b8f2e423490688796a892257e
W20260929 15:55:47.206741 379 leader_election.cc:341] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615): Not found: Tablet not found: 25d0435b8f2e423490688796a892257e
I20260929 15:55:47.206830 379 leader_election.cc:302] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cbc9bc4e27e340ae9b340a346f6041d3; no voters: f84e04a0d55747e2a5d2a6bb4bc77147, fa4812e4098c494994d66a14d5da2638
I20260929 15:55:47.207101 776 raft_consensus.cc:2726] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 15:55:47.597129 777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:47.596431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:53633)
0929 15:55:47.596879 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:47.596966 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":226,"threads_started":1}
I20260929 15:55:47.670660 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:47.669904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:54865)
0929 15:55:47.670376 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:47.670468 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":232,"threads_started":1}
I20260929 15:55:48.137072 779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:48.136358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:41025)
0929 15:55:48.136812 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:48.136897 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":234,"threads_started":1}
I20260929 15:55:48.598832 780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:48.598081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:41921)
0929 15:55:48.598546 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:48.598630 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":250,"threads_started":1}
I20260929 15:55:48.672300 781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:48.671604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:51191)
0929 15:55:48.672014 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:48.672099 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":217,"threads_started":1}
I20260929 15:55:49.138927 782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:49.138118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:41771)
0929 15:55:49.138629 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:49.138724 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":244,"threads_started":1}
I20260929 15:55:49.600555 783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:49.599804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:38763)
0929 15:55:49.600295 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:49.600378 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":221,"threads_started":1}
I20260929 15:55:49.673976 784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:49.673153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:42737)
0929 15:55:49.673714 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:49.673803 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":351,"threads_started":1}
I20260929 15:55:50.140609 785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:50.139920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:58713)
0929 15:55:50.140338 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:50.140434 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":212,"threads_started":1}
I20260929 15:55:50.602051 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:50.601395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:49263)
0929 15:55:50.601796 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:50.601874 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":203,"threads_started":1}
I20260929 15:55:50.631759 787 raft_consensus.cc:482] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:55:50.631942 787 raft_consensus.cc:504] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } }
I20260929 15:55:50.632594 787 leader_election.cc:288] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143), fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615)
W20260929 15:55:50.633432 378 leader_election.cc:341] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143): Not found: Tablet not found: 25d0435b8f2e423490688796a892257e
W20260929 15:55:50.633620 379 leader_election.cc:341] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615): Not found: Tablet not found: 25d0435b8f2e423490688796a892257e
I20260929 15:55:50.633749 379 leader_election.cc:302] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cbc9bc4e27e340ae9b340a346f6041d3; no voters: f84e04a0d55747e2a5d2a6bb4bc77147, fa4812e4098c494994d66a14d5da2638
I20260929 15:55:50.634042 787 raft_consensus.cc:2726] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 15:55:50.675729 788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:50.674849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:33407)
0929 15:55:50.675427 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:50.675546 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":264,"threads_started":1}
I20260929 15:55:51.142297 789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:51.141486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:45745)
0929 15:55:51.142039 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:51.142134 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":318,"threads_started":1}
I20260929 15:55:51.603624 790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:51.602902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:35721)
0929 15:55:51.603389 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:51.603477 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":216,"threads_started":1}
I20260929 15:55:51.677338 791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:51.676664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:48683)
0929 15:55:51.677106 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:51.677181 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":255,"threads_started":1}
I20260929 15:55:52.143893 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:52.143174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:42859)
0929 15:55:52.143630 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:52.143707 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":253,"threads_started":1}
I20260929 15:55:52.555828 793 raft_consensus.cc:482] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:55:52.556068 793 raft_consensus.cc:504] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbc9bc4e27e340ae9b340a346f6041d3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36875 } } peers { permanent_uuid: "f84e04a0d55747e2a5d2a6bb4bc77147" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 38143 } } peers { permanent_uuid: "fa4812e4098c494994d66a14d5da2638" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 33615 } }
I20260929 15:55:52.556794 793 leader_election.cc:288] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143), fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615)
W20260929 15:55:52.557911 378 leader_election.cc:341] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f84e04a0d55747e2a5d2a6bb4bc77147 (127.15.196.195:38143): Not found: Tablet not found: 25d0435b8f2e423490688796a892257e
W20260929 15:55:52.558112 379 leader_election.cc:341] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa4812e4098c494994d66a14d5da2638 (127.15.196.194:33615): Not found: Tablet not found: 25d0435b8f2e423490688796a892257e
I20260929 15:55:52.558221 379 leader_election.cc:302] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cbc9bc4e27e340ae9b340a346f6041d3; no voters: f84e04a0d55747e2a5d2a6bb4bc77147, fa4812e4098c494994d66a14d5da2638
I20260929 15:55:52.558465 793 raft_consensus.cc:2726] T 25d0435b8f2e423490688796a892257e P cbc9bc4e27e340ae9b340a346f6041d3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 15:55:52.605281 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:52.604554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:53599)
0929 15:55:52.605002 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:52.605089 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":248,"threads_started":1}
I20260929 15:55:52.679561 795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:52.678286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:46829)
0929 15:55:52.678994 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:52.679133 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":404,"threads_started":1}
I20260929 15:55:53.145596 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:53.144787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:50219)
0929 15:55:53.145249 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:53.145352 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":217,"threads_started":1}
I20260929 15:55:53.606953 797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:53.606197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:33983)
0929 15:55:53.606690 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:53.606798 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":239,"threads_started":1}
I20260929 15:55:53.681187 798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:53.680393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:53803)
0929 15:55:53.680858 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:53.680945 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":205,"threads_started":1}
I20260929 15:55:54.147435 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:54.146639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:34067)
0929 15:55:54.147096 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:54.147180 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":221,"threads_started":1}
I20260929 15:55:54.608523 800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:54.607852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:45299)
0929 15:55:54.608273 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:54.608354 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":228,"threads_started":1}
I20260929 15:55:54.682823 801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:54.682017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:39507)
0929 15:55:54.682482 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:54.682643 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":252,"threads_started":1}
I20260929 15:55:55.149252 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:55.148421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:50669)
0929 15:55:55.148949 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:55.149079 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":302,"threads_started":1}
I20260929 15:55:55.610226 803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:55.609515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:54995)
0929 15:55:55.609975 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:55.610067 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":234,"threads_started":1}
I20260929 15:55:55.684420 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:55.683723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:60119)
0929 15:55:55.684139 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:55.684226 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":213,"threads_started":1}
I20260929 15:55:56.150776 805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:56.150079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:49511)
0929 15:55:56.150503 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:56.150591 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":230,"threads_started":1}
I20260929 15:55:56.611972 806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:56.611092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:35527)
0929 15:55:56.611675 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:56.611761 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":300,"threads_started":1}
I20260929 15:55:56.686378 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:56.685539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.193:42571)
0929 15:55:56.686079 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:56.686190 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":271,"threads_started":1}
I20260929 15:55:57.152812 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:57.152019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.194:56609)
0929 15:55:57.152532 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:57.152649 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":284,"threads_started":1}
I20260929 15:55:57.596189 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 319
I20260929 15:55:57.611501 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:57.597273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53720 (local address 127.15.196.193:36875)
0929 15:55:57.597743 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:57.597751 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:57.598055 (+ 304us) server_negotiation.cc:407] Connection header received
0929 15:55:57.598310 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:57.598318 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:57.598550 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:57.598726 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:55:57.600230 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:57.601172 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:57.602447 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:57.603022 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:57.603169 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:57.605895 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:55:57.605907 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:55:57.605922 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:55:57.605954 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:55:57.608601 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:57.609280 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:57.609289 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:57.609292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:57.609435 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:57.609826 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:57.609836 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:57.609839 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:57.610198 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:55:57.610503 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:57.610848 (+ 345us) server_negotiation.cc:299] Negotiation successful
0929 15:55:57.611129 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":212,"threads_started":1}
I20260929 15:55:57.612844 486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:55:57.614212 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:57.613177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38147 (local address 127.15.196.195:52549)
0929 15:55:57.613795 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0929 15:55:57.613959 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.254:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":269,"threads_started":1}
I20260929 15:55:57.702023 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 319
I20260929 15:55:57.720099 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 494
I20260929 15:55:57.733314 813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:57.720787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52024 (local address 127.15.196.194:33615)
0929 15:55:57.721242 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:57.721246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:57.721262 (+ 16us) server_negotiation.cc:407] Connection header received
0929 15:55:57.721325 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:57.721328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:57.721386 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:57.721477 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:55:57.722848 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:57.723647 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:57.724952 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:57.725384 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:57.725524 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:57.728222 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:55:57.728227 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:55:57.728231 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:55:57.728263 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:55:57.730661 (+ 2398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:57.731349 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:57.731355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:57.731359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:57.731498 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:57.731863 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:57.731868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:57.731872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:57.732142 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:55:57.732419 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:57.732725 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 15:55:57.732951 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":271,"threads_started":1}
I20260929 15:55:57.734047 617 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:55:57.818239 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 494
I20260929 15:55:57.836102 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 625
I20260929 15:55:57.849822 815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:57.836916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48668 (local address 127.15.196.195:38143)
0929 15:55:57.837480 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:57.837484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:57.837499 (+ 15us) server_negotiation.cc:407] Connection header received
0929 15:55:57.837551 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:57.837554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:57.837616 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:57.837735 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:55:57.839151 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:57.839987 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:57.841190 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:57.841621 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:57.841754 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:57.844544 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:55:57.844549 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:55:57.844552 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:55:57.844582 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:55:57.847090 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:57.847828 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:57.847834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:57.847838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:57.847989 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:57.848360 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:57.848366 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:57.848370 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:57.848630 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:55:57.848919 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:57.849218 (+ 299us) server_negotiation.cc:299] Negotiation successful
0929 15:55:57.849442 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":269,"threads_started":1}
I20260929 15:55:57.850546 748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:55:57.934224 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 625
2026-09-29T15:55:57Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64593 ms)
[----------] 30 tests from SecurityITest (298766 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:55:58 dist-test-slave-c51v krb5kdc[819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:55:58 dist-test-slave-c51v krb5kdc[819](info): set up 2 sockets
Sep 29 15:55:58 dist-test-slave-c51v krb5kdc[819](info): commencing operation
krb5kdc: starting...
W20260929 15:55:59.041487 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:55:59 dist-test-slave-c51v krb5kdc[819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:55:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:55:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:55:59.290417 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:35783
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:40965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:35783
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:55:59.617993 832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:55:59.618335 832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:55:59.618520 832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:55:59.625830 832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:55:59.625974 832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:55:59.626091 832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:55:59.626192 832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:55:59.626278 832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:55:59.636247 832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:40965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:35783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:35783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:55:59.637604 832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:55:59.638981 832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:55:59.650768 838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:59.650862 839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:55:59.650924 841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:55:59.651726 832 server_base.cc:1034] running on GCE node
I20260929 15:55:59.652491 832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:55:59.653856 832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:55:59.655082 832 hybrid_clock.cc:648] HybridClock initialized: now 1790697359655032 us; error 63 us; skew 500 ppm
I20260929 15:55:59.655489 832 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:55:59 dist-test-slave-c51v krb5kdc[819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697359, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:55:59.661710 832 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:55:59.663220 832 webserver.cc:460] Webserver started at http://127.15.196.254:39395/ using document root <none> and password file <none>
I20260929 15:55:59.663939 832 fs_manager.cc:359] Metadata directory not provided
I20260929 15:55:59.664042 832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:55:59.664307 832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:55:59.667238 832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "fe042a04edad46cdb6874b2d4687e3b0"
format_stamp: "Formatted at 2026-09-29 15:55:59 on dist-test-slave-c51v"
server_key: "fd7f6a88ecfe24c5069bf63d5cbff00b"
server_key_iv: "b6836ab45b54dbbf7909060aed103c73"
server_key_version: "encryptionkey@0"
I20260929 15:55:59.668002 832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "fe042a04edad46cdb6874b2d4687e3b0"
format_stamp: "Formatted at 2026-09-29 15:55:59 on dist-test-slave-c51v"
server_key: "fd7f6a88ecfe24c5069bf63d5cbff00b"
server_key_iv: "b6836ab45b54dbbf7909060aed103c73"
server_key_version: "encryptionkey@0"
I20260929 15:55:59.672976 832 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260929 15:55:59.676318 848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:55:59.677629 832 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:55:59.677769 832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "fe042a04edad46cdb6874b2d4687e3b0"
format_stamp: "Formatted at 2026-09-29 15:55:59 on dist-test-slave-c51v"
server_key: "fd7f6a88ecfe24c5069bf63d5cbff00b"
server_key_iv: "b6836ab45b54dbbf7909060aed103c73"
server_key_version: "encryptionkey@0"
I20260929 15:55:59.677927 832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:55:59.702770 832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:55:59.706773 832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:55:59.707006 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:55:59.725147 832 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:35783
I20260929 15:55:59.725167 899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:35783 every 8 connection(s)
I20260929 15:55:59.726562 832 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:55:59.730306 900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:55:59.734612 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 832
I20260929 15:55:59.734848 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:55:59.735694 16147 external_mini_cluster.cc:1428] Setting key d75540a2c6d40eef2cb1dc177695da21
I20260929 15:55:59.742693 900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0: Bootstrap starting.
I20260929 15:55:59.746347 900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0: Neither blocks nor log segments found. Creating new log.
I20260929 15:55:59.747574 900 log.cc:824] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0: Log is configured to *not* fsync() on all Append() calls
I20260929 15:55:59.750203 900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0: No bootstrap required, opened a new log
Sep 29 15:55:59 dist-test-slave-c51v krb5kdc[819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:55:59.759932 903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:55:59.737841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46330 (local address 127.15.196.254:35783)
0929 15:55:59.738614 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0929 15:55:59.738629 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 15:55:59.738672 (+ 43us) server_negotiation.cc:407] Connection header received
0929 15:55:59.740882 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:55:59.740918 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:55:59.741483 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:55:59.741836 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:55:59.743689 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:59.744988 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:59.746978 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:55:59.747749 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:55:59.747874 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:55:59.752512 (+ 4638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:55:59.752520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:55:59.752534 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:55:59.752572 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:55:59.755831 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:59.756651 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:59.756656 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:59.756662 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:59.756815 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:55:59.757219 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:55:59.757222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:55:59.757224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:55:59.757749 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:55:59.757952 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 15:55:59.758450 (+ 498us) server_negotiation.cc:299] Negotiation successful
0929 15:55:59.758830 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":357,"threads_started":1}
I20260929 15:55:59.759835 900 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe042a04edad46cdb6874b2d4687e3b0" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35783 } }
I20260929 15:55:59.760285 900 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:55:59.760423 900 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe042a04edad46cdb6874b2d4687e3b0, State: Initialized, Role: FOLLOWER
I20260929 15:55:59.761019 900 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe042a04edad46cdb6874b2d4687e3b0" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35783 } }
I20260929 15:55:59.761296 900 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:55:59.761443 900 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:55:59.761651 900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:55:59.764308 900 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe042a04edad46cdb6874b2d4687e3b0" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35783 } }
I20260929 15:55:59.764739 900 leader_election.cc:302] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe042a04edad46cdb6874b2d4687e3b0; no voters:
I20260929 15:55:59.765255 900 leader_election.cc:288] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:55:59.765391 905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:55:59.766129 905 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [term 1 LEADER]: Becoming Leader. State: Replica: fe042a04edad46cdb6874b2d4687e3b0, State: Running, Role: LEADER
I20260929 15:55:59.766682 905 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe042a04edad46cdb6874b2d4687e3b0" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35783 } }
I20260929 15:55:59.767429 900 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:55:59.769600 907 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe042a04edad46cdb6874b2d4687e3b0. Latest consensus state: current_term: 1 leader_uuid: "fe042a04edad46cdb6874b2d4687e3b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe042a04edad46cdb6874b2d4687e3b0" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35783 } } }
I20260929 15:55:59.770130 907 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [sys.catalog]: This master's current role is: LEADER
I20260929 15:55:59.769670 906 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe042a04edad46cdb6874b2d4687e3b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe042a04edad46cdb6874b2d4687e3b0" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35783 } } }
I20260929 15:55:59.772989 906 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0 [sys.catalog]: This master's current role is: LEADER
I20260929 15:55:59.780668 914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:55:59.786211 914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:55:59.795485 914 catalog_manager.cc:1347] Generated new cluster ID: 37b948b1f1f74692ac09e1bfcfdb8310
I20260929 15:55:59.795621 914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:55:59.820084 914 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:55:59.821411 914 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:55:59.831254 914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0: Generated new TSK 0
I20260929 15:55:59.832212 914 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:55:59.891907 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:35783
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:40965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:00.240478 927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:00.240777 927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:00.240883 927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:00.248991 927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:00.249138 927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:00.249270 927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:56:00.259473 927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:40965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:35783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:00.260859 927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:00.262117 927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:00.275903 932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:00.275624 933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:00.275786 935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:00.277073 927 server_base.cc:1034] running on GCE node
I20260929 15:56:00.277706 927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:00.278671 927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:00.279908 927 hybrid_clock.cc:648] HybridClock initialized: now 1790697360279873 us; error 48 us; skew 500 ppm
I20260929 15:56:00.280263 927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:00 dist-test-slave-c51v krb5kdc[819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697360, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:00.283542 927 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:56:00.285151 927 webserver.cc:460] Webserver started at http://127.15.196.193:34595/ using document root <none> and password file <none>
I20260929 15:56:00.285842 927 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:00.285987 927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:00.286306 927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:00.289270 927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "804c59155ad74ef09cd75efe8968e336"
format_stamp: "Formatted at 2026-09-29 15:56:00 on dist-test-slave-c51v"
server_key: "b3b9c3441a17fcfa7e5cfd3ca9414237"
server_key_iv: "cec535c85d9db575503eef8db9583fae"
server_key_version: "encryptionkey@0"
I20260929 15:56:00.290047 927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "804c59155ad74ef09cd75efe8968e336"
format_stamp: "Formatted at 2026-09-29 15:56:00 on dist-test-slave-c51v"
server_key: "b3b9c3441a17fcfa7e5cfd3ca9414237"
server_key_iv: "cec535c85d9db575503eef8db9583fae"
server_key_version: "encryptionkey@0"
I20260929 15:56:00.295219 927 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 15:56:00.298521 942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:00.299872 927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 15:56:00.300045 927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "804c59155ad74ef09cd75efe8968e336"
format_stamp: "Formatted at 2026-09-29 15:56:00 on dist-test-slave-c51v"
server_key: "b3b9c3441a17fcfa7e5cfd3ca9414237"
server_key_iv: "cec535c85d9db575503eef8db9583fae"
server_key_version: "encryptionkey@0"
I20260929 15:56:00.300236 927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:00.315632 927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:00.319366 927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:00.319650 927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:00.320624 927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:00.321916 927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:00.321987 927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:00.322135 927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:00.322201 927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:00.354708 927 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:35963
I20260929 15:56:00.354745 1055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:35963 every 8 connection(s)
I20260929 15:56:00.356098 927 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:56:00.356503 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 927
I20260929 15:56:00.356753 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:56:00.357654 16147 external_mini_cluster.cc:1428] Setting key 9993e96e303dd6d05476d716836b681d
Sep 29 15:56:00 dist-test-slave-c51v krb5kdc[819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697360, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:56:00.377929 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:00.358855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:45419 (local address 127.15.196.254:35783)
0929 15:56:00.359368 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:00.359374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:00.360446 (+ 1072us) server_negotiation.cc:407] Connection header received
0929 15:56:00.362268 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:00.362282 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:00.362373 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:00.362491 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:00.364099 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:00.364855 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:00.366399 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:00.367024 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:00.367167 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:00.370706 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:00.370715 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:00.370723 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:00.370766 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:00.374557 (+ 3791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:00.375259 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:00.375266 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:00.375281 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:00.375461 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:00.375908 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:00.375915 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:00.375921 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:00.376297 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:00.376446 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:00.377156 (+ 710us) server_negotiation.cc:299] Negotiation successful
0929 15:56:00.377431 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":240,"threads_started":1}
I20260929 15:56:00.378552 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:00.359562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:35783 (local address 127.15.196.193:45419)
0929 15:56:00.360301 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:00.360338 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:00.362008 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:00.362738 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:00.362747 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:00.363276 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:00.363918 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:00.363936 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:00.365021 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:00.365025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:00.366242 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:00.366254 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:00.367246 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:00.367251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:00.367555 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:00.368299 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:00.368323 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:00.370478 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:00.374690 (+ 4212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:00.374697 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:00.374710 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:00.375122 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:00.375579 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:00.375583 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:00.375585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:00.375767 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:00.376417 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:00.376423 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:00.376740 (+ 317us) client_negotiation.cc:769] Sending connection context
0929 15:56:00.376979 (+ 239us) client_negotiation.cc:240] Negotiation successful
0929 15:56:00.377364 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":323,"threads_started":1}
I20260929 15:56:00.380368 1056 heartbeater.cc:347] Connected to a master server at 127.15.196.254:35783
I20260929 15:56:00.380730 1056 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:00.381719 1056 heartbeater.cc:511] Master 127.15.196.254:35783 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
I20260929 15:56:00.384356 865 ts_manager.cc:194] Registered new tserver with Master: 804c59155ad74ef09cd75efe8968e336 (127.15.196.193:35963)
I20260929 15:56:00.385828 865 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:45419
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:56:00.417932 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:35783
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:40965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:00.750109 1062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:00.750377 1062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:00.750470 1062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:00.757536 1062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:00.757653 1062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:00.757766 1062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:56:00.767567 1062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:40965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:35783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:00.769017 1062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:00.770365 1062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:00.783782 1067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:00.783865 1068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:00.783844 1070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:00.784574 1062 server_base.cc:1034] running on GCE node
I20260929 15:56:00.785156 1062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:00.786078 1062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:00.787356 1062 hybrid_clock.cc:648] HybridClock initialized: now 1790697360787255 us; error 62 us; skew 500 ppm
I20260929 15:56:00.787670 1062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:00 dist-test-slave-c51v krb5kdc[819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697360, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:00.790819 1062 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:56:00.792201 1062 webserver.cc:460] Webserver started at http://127.15.196.194:43511/ using document root <none> and password file <none>
I20260929 15:56:00.792840 1062 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:00.792936 1062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:00.793192 1062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:00.796164 1062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "5972f3d1a8304f43baf1d34bbf2266a6"
format_stamp: "Formatted at 2026-09-29 15:56:00 on dist-test-slave-c51v"
server_key: "6a07218ddc10f99b90b750c3bc0cb43a"
server_key_iv: "04009e5f8fe0e6f6a6ded5d8b7559ac2"
server_key_version: "encryptionkey@0"
I20260929 15:56:00.797020 1062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "5972f3d1a8304f43baf1d34bbf2266a6"
format_stamp: "Formatted at 2026-09-29 15:56:00 on dist-test-slave-c51v"
server_key: "6a07218ddc10f99b90b750c3bc0cb43a"
server_key_iv: "04009e5f8fe0e6f6a6ded5d8b7559ac2"
server_key_version: "encryptionkey@0"
I20260929 15:56:00.802199 1062 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 15:56:00.805860 1077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:00.807184 1062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 15:56:00.807379 1062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "5972f3d1a8304f43baf1d34bbf2266a6"
format_stamp: "Formatted at 2026-09-29 15:56:00 on dist-test-slave-c51v"
server_key: "6a07218ddc10f99b90b750c3bc0cb43a"
server_key_iv: "04009e5f8fe0e6f6a6ded5d8b7559ac2"
server_key_version: "encryptionkey@0"
I20260929 15:56:00.807607 1062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:00.834115 1062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:00.838786 1062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:00.839087 1062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:00.840082 1062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:00.841678 1062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:00.841795 1062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:00.841923 1062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:00.841984 1062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:00.874643 1062 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:43309
I20260929 15:56:00.874680 1190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:43309 every 8 connection(s)
I20260929 15:56:00.876008 1062 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:56:00.882607 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 1062
I20260929 15:56:00.882833 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:56:00.883364 16147 external_mini_cluster.cc:1428] Setting key 402d0ba7f63ad3b1ba9d7ae996269e10
Sep 29 15:56:00 dist-test-slave-c51v krb5kdc[819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697360, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:56:00.897817 1193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:00.878546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:60597 (local address 127.15.196.254:35783)
0929 15:56:00.878958 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:00.878963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:00.880211 (+ 1248us) server_negotiation.cc:407] Connection header received
0929 15:56:00.882204 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:00.882211 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:00.882313 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:00.882460 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:00.884242 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:00.885076 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:00.886788 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:00.887606 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:00.887752 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:00.891360 (+ 3608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:00.891369 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:00.891376 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:00.891421 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:00.894519 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:00.895432 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:00.895437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:00.895439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:00.895575 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:00.896026 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:00.896030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:00.896033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:00.896308 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:00.896471 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:00.897180 (+ 709us) server_negotiation.cc:299] Negotiation successful
0929 15:56:00.897420 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":219,"threads_started":1}
I20260929 15:56:00.898764 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:00.879153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:35783 (local address 127.15.196.194:60597)
0929 15:56:00.880049 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:00.880101 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:00.881867 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:00.882646 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:00.882654 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:00.883331 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:00.884036 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:00.884064 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:00.885240 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:00.885246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:00.886629 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:00.886641 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:00.887812 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:00.887817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:00.888015 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:00.888726 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:00.888749 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:00.891094 (+ 2345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:00.894670 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:00.894679 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:00.894699 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:00.895230 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:00.895692 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:00.895699 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:00.895703 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:00.895897 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:00.896449 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:00.896457 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:00.896788 (+ 331us) client_negotiation.cc:769] Sending connection context
0929 15:56:00.897070 (+ 282us) client_negotiation.cc:240] Negotiation successful
0929 15:56:00.897611 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":338,"threads_started":1}
I20260929 15:56:00.900262 1191 heartbeater.cc:347] Connected to a master server at 127.15.196.254:35783
I20260929 15:56:00.900646 1191 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:00.901525 1191 heartbeater.cc:511] Master 127.15.196.254:35783 requested a full tablet report, sending...
I20260929 15:56:00.903142 865 ts_manager.cc:194] Registered new tserver with Master: 5972f3d1a8304f43baf1d34bbf2266a6 (127.15.196.194:43309)
I20260929 15:56:00.904088 865 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:60597
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:56:00.942620 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:35783
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:40965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:01.271927 1197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:01.272202 1197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:01.272403 1197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:01.279659 1197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:01.279791 1197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:01.279985 1197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:56:01.289870 1197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:40965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:35783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:01.291230 1197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:01.292716 1197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:01.306131 1203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:01.306213 1202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:01.306241 1205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:01.307044 1197 server_base.cc:1034] running on GCE node
I20260929 15:56:01.307710 1197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:01.308655 1197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:01.309885 1197 hybrid_clock.cc:648] HybridClock initialized: now 1790697361309850 us; error 60 us; skew 500 ppm
I20260929 15:56:01.310199 1197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:01 dist-test-slave-c51v krb5kdc[819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697361, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:01.313443 1197 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:56:01.314800 1197 webserver.cc:460] Webserver started at http://127.15.196.195:36335/ using document root <none> and password file <none>
I20260929 15:56:01.315519 1197 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:01.315620 1197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:01.315871 1197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:01.318670 1197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "32c126be4a1d4b1dacb689cc8a1e83a5"
format_stamp: "Formatted at 2026-09-29 15:56:01 on dist-test-slave-c51v"
server_key: "715c6f4058dbe3961a84b30196e675fa"
server_key_iv: "6f5f4341aad099b6e7628d34ec9b65ad"
server_key_version: "encryptionkey@0"
I20260929 15:56:01.319423 1197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "32c126be4a1d4b1dacb689cc8a1e83a5"
format_stamp: "Formatted at 2026-09-29 15:56:01 on dist-test-slave-c51v"
server_key: "715c6f4058dbe3961a84b30196e675fa"
server_key_iv: "6f5f4341aad099b6e7628d34ec9b65ad"
server_key_version: "encryptionkey@0"
I20260929 15:56:01.324982 1197 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260929 15:56:01.328258 1212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:01.329520 1197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:56:01.329648 1197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "32c126be4a1d4b1dacb689cc8a1e83a5"
format_stamp: "Formatted at 2026-09-29 15:56:01 on dist-test-slave-c51v"
server_key: "715c6f4058dbe3961a84b30196e675fa"
server_key_iv: "6f5f4341aad099b6e7628d34ec9b65ad"
server_key_version: "encryptionkey@0"
I20260929 15:56:01.329808 1197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:01.347610 1197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:01.351435 1197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:01.351677 1197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:01.352540 1197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:01.353802 1197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:01.353904 1197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:01.354036 1197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:01.354095 1197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:01.388887 1197 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:36977
I20260929 15:56:01.388959 1056 heartbeater.cc:503] Master 127.15.196.254:35783 was elected leader, sending a full tablet report...
I20260929 15:56:01.389000 1325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:36977 every 8 connection(s)
I20260929 15:56:01.390211 1197 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:56:01.397035 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 1197
I20260929 15:56:01.397245 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:56:01.397776 16147 external_mini_cluster.cc:1428] Setting key 5b76456a72f1c9bc30ae992bbccc5fd0
Sep 29 15:56:01 dist-test-slave-c51v krb5kdc[819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697361, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:56:01.410567 1193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.393000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:39067 (local address 127.15.196.254:35783)
0929 15:56:01.393180 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:01.393185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:01.394270 (+ 1085us) server_negotiation.cc:407] Connection header received
0929 15:56:01.396065 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:01.396070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:01.396139 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:01.396225 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:01.397877 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.398959 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.401160 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.401566 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.401664 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.405119 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:01.405124 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:01.405127 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:01.405162 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:01.407940 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:01.408635 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:01.408639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:01.408641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:01.408762 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:01.409133 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:01.409136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:01.409138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:01.409401 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:01.409515 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:01.410156 (+ 641us) server_negotiation.cc:299] Negotiation successful
0929 15:56:01.410336 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 15:56:01.411700 1328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.393443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:35783 (local address 127.15.196.195:39067)
0929 15:56:01.394111 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:01.394151 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:01.395821 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:01.396440 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:01.396449 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:01.396953 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:01.397696 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:01.397716 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.399171 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.399176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:01.401011 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:01.401024 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.401751 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.401755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:01.401950 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.402573 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:01.402599 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:01.404928 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:01.408069 (+ 3141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:01.408077 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:01.408090 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:01.408525 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:01.408864 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:01.408867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:01.408869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:01.409038 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:01.409501 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:01.409507 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:01.409833 (+ 326us) client_negotiation.cc:769] Sending connection context
0929 15:56:01.410056 (+ 223us) client_negotiation.cc:240] Negotiation successful
0929 15:56:01.410455 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":262,"threads_started":1}
I20260929 15:56:01.413405 1326 heartbeater.cc:347] Connected to a master server at 127.15.196.254:35783
I20260929 15:56:01.413718 1326 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:01.414458 1326 heartbeater.cc:511] Master 127.15.196.254:35783 requested a full tablet report, sending...
I20260929 15:56:01.415934 865 ts_manager.cc:194] Registered new tserver with Master: 32c126be4a1d4b1dacb689cc8a1e83a5 (127.15.196.195:36977)
I20260929 15:56:01.416694 865 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:39067
I20260929 15:56:01.425824 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:56:01.442828 1193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.430205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.15.196.254:35783)
0929 15:56:01.430434 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:01.430444 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:01.430757 (+ 313us) server_negotiation.cc:407] Connection header received
0929 15:56:01.430917 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:01.430921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:01.430981 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:01.431058 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:01.432551 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.433277 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.434618 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.435035 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.435170 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.437585 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:01.437592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:01.437598 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:01.437634 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:01.440204 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:01.440845 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:01.440852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:01.440858 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:01.440979 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:01.441357 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:01.441362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:01.441365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:01.441664 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:01.441937 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:01.442262 (+ 325us) server_negotiation.cc:299] Negotiation successful
0929 15:56:01.442494 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 15:56:01.446707 865 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:56:01.453684 865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:56:01.480377 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.465106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:36977 (local address 127.0.0.1:48292)
0929 15:56:01.466438 (+ 1332us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:01.466476 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:01.466612 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:01.467754 (+ 1142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:01.467765 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:01.467802 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:01.468257 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:01.468278 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.470998 (+ 2720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.471007 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:01.474776 (+ 3769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:01.474799 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.477751 (+ 2952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.477760 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:01.478051 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.478123 (+ 72us) client_negotiation.cc:769] Sending connection context
0929 15:56:01.478289 (+ 166us) client_negotiation.cc:240] Negotiation successful
0929 15:56:01.478593 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1123,"thread_start_us":1025,"threads_started":1}
I20260929 15:56:01.481114 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.465314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48292 (local address 127.15.196.195:36977)
0929 15:56:01.466213 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:01.466222 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:01.466576 (+ 354us) server_negotiation.cc:407] Connection header received
0929 15:56:01.466799 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:01.466809 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:01.467140 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:01.467482 (+ 342us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:01.468591 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.470792 (+ 2201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.475024 (+ 4232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.477055 (+ 2031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.477295 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.477559 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:01.479355 (+ 1796us) server_negotiation.cc:299] Negotiation successful
0929 15:56:01.479606 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":458,"threads_started":1}
I20260929 15:56:01.482564 1340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.466938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:43309 (local address 127.0.0.1:47882)
0929 15:56:01.467470 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:01.467492 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:01.467628 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:01.470219 (+ 2591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:01.470226 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:01.470261 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:01.470772 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:01.470786 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.474955 (+ 4169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.474964 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:01.477901 (+ 2937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:01.477921 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.481545 (+ 3624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.481554 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:01.481821 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.481873 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 15:56:01.481994 (+ 121us) client_negotiation.cc:240] Negotiation successful
0929 15:56:01.482205 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":236,"threads_started":1}
I20260929 15:56:01.483183 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.467171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47882 (local address 127.15.196.194:43309)
0929 15:56:01.469443 (+ 2272us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:01.469457 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:01.469492 (+ 35us) server_negotiation.cc:407] Connection header received
0929 15:56:01.469623 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:01.469634 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:01.469962 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:01.471017 (+ 1055us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:01.471278 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.474749 (+ 3471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.478908 (+ 4159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.481261 (+ 2353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.481511 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.481815 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:01.482850 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0929 15:56:01.482984 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":713,"threads_started":1}
I20260929 15:56:01.486382 1260 tablet_service.cc:1467] Processing CreateTablet for tablet 0ccf371584034488be92bf6269f56787 (DEFAULT_TABLE table=test-table [id=2b3cdd3ee6b949adbff61d0b1e7a9818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:01.486954 1125 tablet_service.cc:1467] Processing CreateTablet for tablet 0ccf371584034488be92bf6269f56787 (DEFAULT_TABLE table=test-table [id=2b3cdd3ee6b949adbff61d0b1e7a9818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:01.487843 1260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ccf371584034488be92bf6269f56787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:01.488298 1125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ccf371584034488be92bf6269f56787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:01.490466 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.467496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:35963 (local address 127.0.0.1:46672)
0929 15:56:01.480664 (+ 13168us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:01.480682 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:01.480796 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:01.483525 (+ 2729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:01.483529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:01.483555 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:01.483726 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:01.483737 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.485841 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.485846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:01.487657 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:01.487676 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.489720 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.489728 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:01.489930 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.489961 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 15:56:01.490041 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 15:56:01.490175 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13077,"thread_start_us":14579,"threads_started":1}
I20260929 15:56:01.491231 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.468008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46672 (local address 127.15.196.193:35963)
0929 15:56:01.482713 (+ 14705us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:01.482723 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:01.482751 (+ 28us) server_negotiation.cc:407] Connection header received
0929 15:56:01.482859 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:01.482867 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:01.483123 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:01.483534 (+ 411us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:01.484059 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.485671 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.487851 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.489253 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.489457 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.489616 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:01.490729 (+ 1113us) server_negotiation.cc:299] Negotiation successful
0929 15:56:01.490915 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14524,"thread_start_us":14368,"threads_started":1}
I20260929 15:56:01.494940 991 tablet_service.cc:1467] Processing CreateTablet for tablet 0ccf371584034488be92bf6269f56787 (DEFAULT_TABLE table=test-table [id=2b3cdd3ee6b949adbff61d0b1e7a9818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:01.496505 991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ccf371584034488be92bf6269f56787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:01.500267 1344 tablet_bootstrap.cc:492] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5: Bootstrap starting.
I20260929 15:56:01.502509 1345 tablet_bootstrap.cc:492] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6: Bootstrap starting.
I20260929 15:56:01.503931 1344 tablet_bootstrap.cc:654] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:01.505316 1344 log.cc:824] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:01.505921 1345 tablet_bootstrap.cc:654] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:01.507445 1345 log.cc:824] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:01.510314 1345 tablet_bootstrap.cc:492] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6: No bootstrap required, opened a new log
I20260929 15:56:01.510609 1345 ts_tablet_manager.cc:1397] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 15:56:01.510777 1344 tablet_bootstrap.cc:492] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5: No bootstrap required, opened a new log
I20260929 15:56:01.511124 1344 ts_tablet_manager.cc:1397] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20260929 15:56:01.511693 1348 tablet_bootstrap.cc:492] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336: Bootstrap starting.
I20260929 15:56:01.515105 1348 tablet_bootstrap.cc:654] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:01.516275 1348 log.cc:824] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:01.519434 1344 raft_consensus.cc:348] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c126be4a1d4b1dacb689cc8a1e83a5" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 36977 } } peers { permanent_uuid: "5972f3d1a8304f43baf1d34bbf2266a6" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 43309 } } peers { permanent_uuid: "804c59155ad74ef09cd75efe8968e336" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35963 } }
I20260929 15:56:01.519830 1344 raft_consensus.cc:374] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:01.519929 1344 raft_consensus.cc:729] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c126be4a1d4b1dacb689cc8a1e83a5, State: Initialized, Role: FOLLOWER
I20260929 15:56:01.519949 1348 tablet_bootstrap.cc:492] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336: No bootstrap required, opened a new log
I20260929 15:56:01.520218 1348 ts_tablet_manager.cc:1397] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 15:56:01.520552 1344 consensus_queue.cc:260] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c126be4a1d4b1dacb689cc8a1e83a5" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 36977 } } peers { permanent_uuid: "5972f3d1a8304f43baf1d34bbf2266a6" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 43309 } } peers { permanent_uuid: "804c59155ad74ef09cd75efe8968e336" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35963 } }
I20260929 15:56:01.521237 1345 raft_consensus.cc:348] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c126be4a1d4b1dacb689cc8a1e83a5" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 36977 } } peers { permanent_uuid: "5972f3d1a8304f43baf1d34bbf2266a6" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 43309 } } peers { permanent_uuid: "804c59155ad74ef09cd75efe8968e336" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35963 } }
I20260929 15:56:01.521660 1345 raft_consensus.cc:374] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:01.521754 1345 raft_consensus.cc:729] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5972f3d1a8304f43baf1d34bbf2266a6, State: Initialized, Role: FOLLOWER
I20260929 15:56:01.522298 1345 consensus_queue.cc:260] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c126be4a1d4b1dacb689cc8a1e83a5" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 36977 } } peers { permanent_uuid: "5972f3d1a8304f43baf1d34bbf2266a6" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 43309 } } peers { permanent_uuid: "804c59155ad74ef09cd75efe8968e336" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35963 } }
I20260929 15:56:01.524107 1191 heartbeater.cc:503] Master 127.15.196.254:35783 was elected leader, sending a full tablet report...
I20260929 15:56:01.524662 1345 ts_tablet_manager.cc:1428] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260929 15:56:01.527915 1326 heartbeater.cc:503] Master 127.15.196.254:35783 was elected leader, sending a full tablet report...
I20260929 15:56:01.528599 1344 ts_tablet_manager.cc:1428] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5: Time spent starting tablet: real 0.017s user 0.010s sys 0.007s
I20260929 15:56:01.530819 1348 raft_consensus.cc:348] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c126be4a1d4b1dacb689cc8a1e83a5" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 36977 } } peers { permanent_uuid: "5972f3d1a8304f43baf1d34bbf2266a6" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 43309 } } peers { permanent_uuid: "804c59155ad74ef09cd75efe8968e336" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35963 } }
I20260929 15:56:01.531270 1348 raft_consensus.cc:374] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:01.531410 1348 raft_consensus.cc:729] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 804c59155ad74ef09cd75efe8968e336, State: Initialized, Role: FOLLOWER
I20260929 15:56:01.532065 1348 consensus_queue.cc:260] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c126be4a1d4b1dacb689cc8a1e83a5" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 36977 } } peers { permanent_uuid: "5972f3d1a8304f43baf1d34bbf2266a6" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 43309 } } peers { permanent_uuid: "804c59155ad74ef09cd75efe8968e336" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35963 } }
I20260929 15:56:01.533730 1348 ts_tablet_manager.cc:1428] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260929 15:56:01.551048 1350 raft_consensus.cc:482] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:56:01.551411 1350 raft_consensus.cc:504] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c126be4a1d4b1dacb689cc8a1e83a5" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 36977 } } peers { permanent_uuid: "5972f3d1a8304f43baf1d34bbf2266a6" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 43309 } } peers { permanent_uuid: "804c59155ad74ef09cd75efe8968e336" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35963 } }
I20260929 15:56:01.553081 1350 leader_election.cc:288] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5972f3d1a8304f43baf1d34bbf2266a6 (127.15.196.194:43309), 804c59155ad74ef09cd75efe8968e336 (127.15.196.193:35963)
I20260929 15:56:01.559690 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.553632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:58387 (local address 127.15.196.194:43309)
0929 15:56:01.553884 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:01.553891 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:01.553914 (+ 23us) server_negotiation.cc:407] Connection header received
0929 15:56:01.553987 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:01.553993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:01.554070 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:01.554167 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:01.554705 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.555717 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.557675 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.558773 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.558892 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.558946 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:01.559369 (+ 423us) server_negotiation.cc:299] Negotiation successful
0929 15:56:01.559464 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260929 15:56:01.559762 1328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.553412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:43309 (local address 127.15.196.195:58387)
0929 15:56:01.553616 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:01.553638 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:01.553751 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:01.554303 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:01.554309 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:01.554340 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:01.554523 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:01.554533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.555877 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.555883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:01.557508 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:01.557520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.558931 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.558937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:01.559132 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.559182 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 15:56:01.559310 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 15:56:01.559454 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":46}
I20260929 15:56:01.560514 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.553998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:35833 (local address 127.15.196.193:35963)
0929 15:56:01.554202 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:01.554208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:01.554539 (+ 331us) server_negotiation.cc:407] Connection header received
0929 15:56:01.554678 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:01.554683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:01.554739 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:01.554805 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:01.555462 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.556478 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.558438 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.559532 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.559700 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.559778 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:01.560149 (+ 371us) server_negotiation.cc:299] Negotiation successful
0929 15:56:01.560278 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 15:56:01.560638 1353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.553769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:35963 (local address 127.15.196.195:35833)
0929 15:56:01.554435 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:01.554450 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:01.554545 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:01.554846 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:01.554850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:01.554873 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:01.555252 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:01.555262 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.556630 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.556634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:01.558264 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:01.558278 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.559751 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.559758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:01.559954 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.560002 (+ 48us) client_negotiation.cc:769] Sending connection context
0929 15:56:01.560122 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 15:56:01.560253 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":1,"thread_start_us":341,"threads_started":1}
I20260929 15:56:01.560825 1146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ccf371584034488be92bf6269f56787" candidate_uuid: "32c126be4a1d4b1dacb689cc8a1e83a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5972f3d1a8304f43baf1d34bbf2266a6" is_pre_election: true
I20260929 15:56:01.561422 1146 raft_consensus.cc:2454] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32c126be4a1d4b1dacb689cc8a1e83a5 in term 0.
I20260929 15:56:01.561646 1011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ccf371584034488be92bf6269f56787" candidate_uuid: "32c126be4a1d4b1dacb689cc8a1e83a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "804c59155ad74ef09cd75efe8968e336" is_pre_election: true
I20260929 15:56:01.562091 1214 leader_election.cc:302] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c126be4a1d4b1dacb689cc8a1e83a5, 5972f3d1a8304f43baf1d34bbf2266a6; no voters:
I20260929 15:56:01.562129 1011 raft_consensus.cc:2454] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32c126be4a1d4b1dacb689cc8a1e83a5 in term 0.
I20260929 15:56:01.562477 1350 raft_consensus.cc:2781] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:56:01.562650 1350 raft_consensus.cc:482] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:56:01.562770 1350 raft_consensus.cc:3037] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:01.565203 1350 raft_consensus.cc:504] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c126be4a1d4b1dacb689cc8a1e83a5" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 36977 } } peers { permanent_uuid: "5972f3d1a8304f43baf1d34bbf2266a6" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 43309 } } peers { permanent_uuid: "804c59155ad74ef09cd75efe8968e336" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35963 } }
I20260929 15:56:01.565922 1350 leader_election.cc:288] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [CANDIDATE]: Term 1 election: Requested vote from peers 5972f3d1a8304f43baf1d34bbf2266a6 (127.15.196.194:43309), 804c59155ad74ef09cd75efe8968e336 (127.15.196.193:35963)
I20260929 15:56:01.566325 1146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ccf371584034488be92bf6269f56787" candidate_uuid: "32c126be4a1d4b1dacb689cc8a1e83a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5972f3d1a8304f43baf1d34bbf2266a6"
I20260929 15:56:01.566412 1011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ccf371584034488be92bf6269f56787" candidate_uuid: "32c126be4a1d4b1dacb689cc8a1e83a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "804c59155ad74ef09cd75efe8968e336"
I20260929 15:56:01.566531 1146 raft_consensus.cc:3037] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:01.566634 1011 raft_consensus.cc:3037] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:01.569139 1011 raft_consensus.cc:2454] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c126be4a1d4b1dacb689cc8a1e83a5 in term 1.
I20260929 15:56:01.569139 1146 raft_consensus.cc:2454] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32c126be4a1d4b1dacb689cc8a1e83a5 in term 1.
I20260929 15:56:01.569526 1214 leader_election.cc:302] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32c126be4a1d4b1dacb689cc8a1e83a5, 804c59155ad74ef09cd75efe8968e336; no voters:
I20260929 15:56:01.569844 1350 raft_consensus.cc:2781] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:01.570288 1350 raft_consensus.cc:686] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [term 1 LEADER]: Becoming Leader. State: Replica: 32c126be4a1d4b1dacb689cc8a1e83a5, State: Running, Role: LEADER
I20260929 15:56:01.570773 1350 consensus_queue.cc:237] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c126be4a1d4b1dacb689cc8a1e83a5" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 36977 } } peers { permanent_uuid: "5972f3d1a8304f43baf1d34bbf2266a6" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 43309 } } peers { permanent_uuid: "804c59155ad74ef09cd75efe8968e336" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35963 } }
I20260929 15:56:01.575234 864 catalog_manager.cc:5495] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32c126be4a1d4b1dacb689cc8a1e83a5 (127.15.196.195). New cstate: current_term: 1 leader_uuid: "32c126be4a1d4b1dacb689cc8a1e83a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32c126be4a1d4b1dacb689cc8a1e83a5" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 36977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5972f3d1a8304f43baf1d34bbf2266a6" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 43309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "804c59155ad74ef09cd75efe8968e336" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35963 } health_report { overall_health: UNKNOWN } } }
I20260929 15:56:01.601646 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.596432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48298 (local address 127.15.196.195:36977)
0929 15:56:01.596618 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:01.596622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:01.596640 (+ 18us) server_negotiation.cc:407] Connection header received
0929 15:56:01.596788 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:01.596792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:01.596870 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:01.596996 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:01.597322 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.598149 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.599648 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.600045 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.600139 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.600559 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:01.601080 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:01.601196 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:01.601283 (+ 87us) server_negotiation.cc:299] Negotiation successful
0929 15:56:01.601350 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260929 15:56:01.608659 1057 tablet.cc:2369] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:56:01.610257 1011 raft_consensus.cc:1261] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336 [term 1 FOLLOWER]: Refusing update from remote peer 32c126be4a1d4b1dacb689cc8a1e83a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:56:01.610258 1146 raft_consensus.cc:1261] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6 [term 1 FOLLOWER]: Refusing update from remote peer 32c126be4a1d4b1dacb689cc8a1e83a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:56:01.611282 1354 consensus_queue.cc:1035] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "804c59155ad74ef09cd75efe8968e336" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:56:01.611694 1350 consensus_queue.cc:1035] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5972f3d1a8304f43baf1d34bbf2266a6" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 43309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:56:01.626931 1359 mvcc.cc:205] Tried to move back new op lower bound from 7334696393144807424 to 7334696392997773312. Current Snapshot: MvccSnapshot[applied={T|T < 7334696393144807424}]
I20260929 15:56:01.636860 864 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:56:01.637137 864 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:56:01.640597 864 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fe042a04edad46cdb6874b2d4687e3b0: Sending DeleteTablet for 3 replicas of tablet 0ccf371584034488be92bf6269f56787
I20260929 15:56:01.641686 1260 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ccf371584034488be92bf6269f56787 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:01 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:48292
I20260929 15:56:01.641819 991 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ccf371584034488be92bf6269f56787 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:01 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:46672
I20260929 15:56:01.642668 1366 tablet_replica.cc:330] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5: stopping tablet replica
I20260929 15:56:01.642798 1367 tablet_replica.cc:330] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336: stopping tablet replica
I20260929 15:56:01.643190 1367 raft_consensus.cc:2229] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:56:01.643275 1125 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ccf371584034488be92bf6269f56787 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:01 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:47882
I20260929 15:56:01.643714 1367 raft_consensus.cc:2258] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:01.643754 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 927
I20260929 15:56:01.645792 1366 raft_consensus.cc:2229] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:56:01.646312 1366 raft_consensus.cc:2258] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:01.647948 1367 ts_tablet_manager.cc:1905] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:56:01.653152 1366 ts_tablet_manager.cc:1905] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:56:01.654343 1367 ts_tablet_manager.cc:1918] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:56:01.654524 1367 log.cc:1196] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/0ccf371584034488be92bf6269f56787
I20260929 15:56:01.655143 1367 ts_tablet_manager.cc:1939] T 0ccf371584034488be92bf6269f56787 P 804c59155ad74ef09cd75efe8968e336: Deleting consensus metadata
I20260929 15:56:01.657171 850 catalog_manager.cc:4841] TS 804c59155ad74ef09cd75efe8968e336 (127.15.196.193:35963): tablet 0ccf371584034488be92bf6269f56787 (table test-table [id=2b3cdd3ee6b949adbff61d0b1e7a9818]) successfully deleted
Sep 29 15:56:01 dist-test-slave-c51v krb5kdc[819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:56:01.657987 1368 tablet_replica.cc:330] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6: stopping tablet replica
I20260929 15:56:01.658501 1368 raft_consensus.cc:2229] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:56:01.659066 1368 raft_consensus.cc:2258] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:01.661618 1368 ts_tablet_manager.cc:1905] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:56:01.662452 1366 ts_tablet_manager.cc:1918] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:56:01.662669 1366 log.cc:1196] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/0ccf371584034488be92bf6269f56787
I20260929 15:56:01.663308 1366 ts_tablet_manager.cc:1939] T 0ccf371584034488be92bf6269f56787 P 32c126be4a1d4b1dacb689cc8a1e83a5: Deleting consensus metadata
I20260929 15:56:01.665078 850 catalog_manager.cc:4841] TS 32c126be4a1d4b1dacb689cc8a1e83a5 (127.15.196.195:36977): tablet 0ccf371584034488be92bf6269f56787 (table test-table [id=2b3cdd3ee6b949adbff61d0b1e7a9818]) successfully deleted
I20260929 15:56:01.666352 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.644948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46684 (local address 127.15.196.193:35963)
0929 15:56:01.645118 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:01.645126 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:01.645579 (+ 453us) server_negotiation.cc:407] Connection header received
0929 15:56:01.645746 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:01.645754 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:01.645868 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:01.646001 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:01.648268 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.649690 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.651742 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.652752 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.652915 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.658762 (+ 5847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:01.658776 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:01.658793 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:01.658845 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:01.662742 (+ 3897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:01.663648 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:01.663657 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:01.663662 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:01.663826 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:01.664282 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:01.664289 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:01.664294 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:01.664746 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:01.665042 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:01.665718 (+ 676us) server_negotiation.cc:299] Negotiation successful
0929 15:56:01.665953 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 15:56:01.667013 1051 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:01.669159 1368 ts_tablet_manager.cc:1918] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:56:01.669337 1368 log.cc:1196] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/0ccf371584034488be92bf6269f56787
I20260929 15:56:01.669817 1368 ts_tablet_manager.cc:1939] T 0ccf371584034488be92bf6269f56787 P 5972f3d1a8304f43baf1d34bbf2266a6: Deleting consensus metadata
I20260929 15:56:01.671386 850 catalog_manager.cc:4841] TS 5972f3d1a8304f43baf1d34bbf2266a6 (127.15.196.194:43309): tablet 0ccf371584034488be92bf6269f56787 (table test-table [id=2b3cdd3ee6b949adbff61d0b1e7a9818]) successfully deleted
I20260929 15:56:01.760808 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 927
I20260929 15:56:01.780120 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 1062
Sep 29 15:56:01 dist-test-slave-c51v krb5kdc[819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:56:01.799386 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.780984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47890 (local address 127.15.196.194:43309)
0929 15:56:01.781186 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:01.781193 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:01.781379 (+ 186us) server_negotiation.cc:407] Connection header received
0929 15:56:01.781632 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:01.781638 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:01.781736 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:01.781880 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:01.783762 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.785028 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.786957 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.787442 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.787553 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.792620 (+ 5067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:01.792641 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:01.792661 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:01.792715 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:01.796408 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:01.797088 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:01.797098 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:01.797103 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:01.797253 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:01.797601 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:01.797606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:01.797610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:01.798097 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:01.798430 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:01.798751 (+ 321us) server_negotiation.cc:299] Negotiation successful
0929 15:56:01.798985 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 15:56:01.800226 1186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:01.890491 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 1062
I20260929 15:56:01.910465 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 1197
Sep 29 15:56:01 dist-test-slave-c51v krb5kdc[819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:56:01.929091 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:01.911530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48302 (local address 127.15.196.195:36977)
0929 15:56:01.911754 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:01.911763 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:01.911795 (+ 32us) server_negotiation.cc:407] Connection header received
0929 15:56:01.911879 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:01.911885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:01.912003 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:01.912133 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:01.913471 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.914770 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.916306 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:01.917029 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:01.917198 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:01.921555 (+ 4357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:01.921569 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:01.921590 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:01.921688 (+ 98us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:01.925731 (+ 4043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:01.926558 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:01.926569 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:01.926580 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:01.926795 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:01.927207 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:01.927213 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:01.927217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:01.927780 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:01.928134 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:01.928344 (+ 210us) server_negotiation.cc:299] Negotiation successful
0929 15:56:01.928593 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 15:56:01.930048 1321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:02.051409 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 1197
I20260929 15:56:02.071118 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 832
I20260929 15:56:02.072274 895 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:02.185698 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 832
2026-09-29T15:56:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4258 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-29T15:56:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:02Z Disabled control of system clock
I20260929 15:56:02.266443 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:45959
--webserver_interface=127.15.196.254
--webserver_port=0
--builtin_ntp_servers=127.15.196.212:44681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:45959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 15:56:02.595633 1377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:02.595989 1377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:02.596135 1377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:02.603675 1377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:02.603793 1377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:02.603865 1377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:02.603951 1377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:02.604032 1377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:02.613453 1377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:44681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:45959
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:45959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:02.614831 1377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:02.616658 1377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:02.628091 1385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:02.628130 1383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:02.628091 1382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:02.629110 1377 server_base.cc:1034] running on GCE node
I20260929 15:56:02.629796 1377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:02.631156 1377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:02.632390 1377 hybrid_clock.cc:648] HybridClock initialized: now 1790697362632343 us; error 40 us; skew 500 ppm
I20260929 15:56:02.632750 1377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:56:02.634984 1377 webserver.cc:460] Webserver started at http://127.15.196.254:40887/ using document root <none> and password file <none>
I20260929 15:56:02.635736 1377 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:02.635875 1377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:02.636199 1377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:02.639225 1377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "5695a979aba44ab39b3f26743c352b46"
format_stamp: "Formatted at 2026-09-29 15:56:02 on dist-test-slave-c51v"
server_key: "10110877cb0f7f306407599d2f8b0145"
server_key_iv: "cba927857bd13dd2b7be85e1ff5797a2"
server_key_version: "encryptionkey@0"
I20260929 15:56:02.640080 1377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "5695a979aba44ab39b3f26743c352b46"
format_stamp: "Formatted at 2026-09-29 15:56:02 on dist-test-slave-c51v"
server_key: "10110877cb0f7f306407599d2f8b0145"
server_key_iv: "cba927857bd13dd2b7be85e1ff5797a2"
server_key_version: "encryptionkey@0"
I20260929 15:56:02.645066 1377 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:56:02.648569 1391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:02.650048 1377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:56:02.650245 1377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "5695a979aba44ab39b3f26743c352b46"
format_stamp: "Formatted at 2026-09-29 15:56:02 on dist-test-slave-c51v"
server_key: "10110877cb0f7f306407599d2f8b0145"
server_key_iv: "cba927857bd13dd2b7be85e1ff5797a2"
server_key_version: "encryptionkey@0"
I20260929 15:56:02.650472 1377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:02.671568 1377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:02.672458 1377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:02.672762 1377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:02.692412 1377 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:45959
I20260929 15:56:02.692436 1443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:45959 every 8 connection(s)
I20260929 15:56:02.693859 1377 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:02.697666 1444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:02.700004 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 1377
I20260929 15:56:02.700206 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:02.700708 16147 external_mini_cluster.cc:1428] Setting key 3a3b225de125551a4e2d73b705a12b6f
I20260929 15:56:02.711337 1444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46: Bootstrap starting.
I20260929 15:56:02.715250 1444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:02.716588 1447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:02.702679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58552 (local address 127.15.196.254:45959)
0929 15:56:02.703785 (+ 1106us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:02.703801 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:02.703854 (+ 53us) server_negotiation.cc:407] Connection header received
0929 15:56:02.706237 (+ 2383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:02.706274 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:02.706890 (+ 616us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:02.707396 (+ 506us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:02.707975 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:02.710029 (+ 2054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:02.711460 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:02.712443 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:02.712621 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:02.713740 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:02.713750 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:02.713767 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:02.714488 (+ 721us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:02.714602 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:02.714633 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:02.714754 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:02.714947 (+ 193us) server_negotiation.cc:299] Negotiation successful
0929 15:56:02.715307 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":355,"threads_started":1}
I20260929 15:56:02.716576 1444 log.cc:824] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:02.719468 1444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46: No bootstrap required, opened a new log
I20260929 15:56:02.726990 1444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695a979aba44ab39b3f26743c352b46" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45959 } }
I20260929 15:56:02.727378 1444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:02.727487 1444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5695a979aba44ab39b3f26743c352b46, State: Initialized, Role: FOLLOWER
I20260929 15:56:02.728108 1444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695a979aba44ab39b3f26743c352b46" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45959 } }
I20260929 15:56:02.728349 1444 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:02.728475 1444 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:02.728655 1444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:02.730822 1444 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695a979aba44ab39b3f26743c352b46" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45959 } }
I20260929 15:56:02.731428 1444 leader_election.cc:302] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5695a979aba44ab39b3f26743c352b46; no voters:
I20260929 15:56:02.731974 1444 leader_election.cc:288] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:02.732125 1449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:02.732838 1449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [term 1 LEADER]: Becoming Leader. State: Replica: 5695a979aba44ab39b3f26743c352b46, State: Running, Role: LEADER
I20260929 15:56:02.733310 1449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695a979aba44ab39b3f26743c352b46" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45959 } }
I20260929 15:56:02.734254 1444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:02.736136 1451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5695a979aba44ab39b3f26743c352b46. Latest consensus state: current_term: 1 leader_uuid: "5695a979aba44ab39b3f26743c352b46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695a979aba44ab39b3f26743c352b46" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45959 } } }
I20260929 15:56:02.736581 1451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:02.736965 1450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5695a979aba44ab39b3f26743c352b46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695a979aba44ab39b3f26743c352b46" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45959 } } }
I20260929 15:56:02.737282 1450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:02.738811 1456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:02.743984 1456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:02.753643 1456 catalog_manager.cc:1347] Generated new cluster ID: 62fd2296832447839026a9cb2a8629ab
I20260929 15:56:02.753773 1456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:02.772284 1456 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:02.773608 1456 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:02.783087 1456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46: Generated new TSK 0
I20260929 15:56:02.783990 1456 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:02.799878 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:45959
--builtin_ntp_servers=127.15.196.212:44681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 15:56:03.176746 1468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:03.177122 1468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:03.177269 1468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:03.187520 1468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:03.187649 1468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:03.187819 1468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:56:03.198307 1468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:44681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:45959
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:03.199774 1468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:03.201664 1468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:03.214496 1473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:03.214596 1474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:03.214613 1476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:03.216150 1468 server_base.cc:1034] running on GCE node
I20260929 15:56:03.216761 1468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:03.217721 1468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:03.218951 1468 hybrid_clock.cc:648] HybridClock initialized: now 1790697363218900 us; error 64 us; skew 500 ppm
I20260929 15:56:03.219267 1468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:56:03.221513 1468 webserver.cc:460] Webserver started at http://127.15.196.193:39773/ using document root <none> and password file <none>
I20260929 15:56:03.222195 1468 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:03.222328 1468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:03.222657 1468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:03.225533 1468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "f3bd322ac37e4079b32ab3291a7947f3"
format_stamp: "Formatted at 2026-09-29 15:56:03 on dist-test-slave-c51v"
server_key: "ef9a5afeddd5cff3230d598072a0c652"
server_key_iv: "818a9db8d95d9415863dbb384d583ce4"
server_key_version: "encryptionkey@0"
I20260929 15:56:03.226325 1468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "f3bd322ac37e4079b32ab3291a7947f3"
format_stamp: "Formatted at 2026-09-29 15:56:03 on dist-test-slave-c51v"
server_key: "ef9a5afeddd5cff3230d598072a0c652"
server_key_iv: "818a9db8d95d9415863dbb384d583ce4"
server_key_version: "encryptionkey@0"
I20260929 15:56:03.231550 1468 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260929 15:56:03.234896 1482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:03.236374 1468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:03.236554 1468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "f3bd322ac37e4079b32ab3291a7947f3"
format_stamp: "Formatted at 2026-09-29 15:56:03 on dist-test-slave-c51v"
server_key: "ef9a5afeddd5cff3230d598072a0c652"
server_key_iv: "818a9db8d95d9415863dbb384d583ce4"
server_key_version: "encryptionkey@0"
I20260929 15:56:03.236775 1468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:03.254483 1468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:03.255398 1468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:03.255712 1468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:03.256639 1468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:03.258065 1468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:03.258189 1468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:03.258325 1468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:03.258417 1468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:03.291558 1468 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:35933
I20260929 15:56:03.291610 1594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:35933 every 8 connection(s)
I20260929 15:56:03.292966 1468 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:56:03.301462 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 1468
I20260929 15:56:03.301669 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:56:03.302114 16147 external_mini_cluster.cc:1428] Setting key c5b070d4f7ffe5d9092773aa588aec78
I20260929 15:56:03.304982 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:45959
--builtin_ntp_servers=127.15.196.212:44681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 15:56:03.306116 1597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:03.295576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:46683 (local address 127.15.196.254:45959)
0929 15:56:03.296059 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:03.296064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:03.297122 (+ 1058us) server_negotiation.cc:407] Connection header received
0929 15:56:03.298954 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:03.298960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:03.299049 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:03.299167 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:03.300268 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:03.301008 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:03.302891 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:03.303407 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:03.303508 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:03.305126 (+ 1618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:03.305134 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:03.305141 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:03.305168 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:03.305228 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:03.305248 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:03.305342 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:03.305557 (+ 215us) server_negotiation.cc:299] Negotiation successful
0929 15:56:03.305667 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":276,"threads_started":1}
I20260929 15:56:03.306910 1598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:03.296098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:45959 (local address 127.15.196.193:46683)
0929 15:56:03.296966 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:03.297001 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:03.298684 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:03.299440 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:03.299447 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:03.299535 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:03.300100 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:03.300115 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:03.301165 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:03.301170 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:03.302736 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:03.302752 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:03.303617 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:03.303624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:03.303960 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:03.304792 (+ 832us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:56:03.304794 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:03.304840 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:56:03.304919 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:03.305353 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:03.305362 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:03.305368 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 15:56:03.305512 (+ 144us) client_negotiation.cc:240] Negotiation successful
0929 15:56:03.305819 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":258,"threads_started":1}
I20260929 15:56:03.308495 1595 heartbeater.cc:347] Connected to a master server at 127.15.196.254:45959
I20260929 15:56:03.308831 1595 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:03.309754 1595 heartbeater.cc:511] Master 127.15.196.254:45959 requested a full tablet report, sending...
I20260929 15:56:03.311900 1408 ts_manager.cc:194] Registered new tserver with Master: f3bd322ac37e4079b32ab3291a7947f3 (127.15.196.193:35933)
I20260929 15:56:03.313364 1408 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.196.193:46683
W20260929 15:56:03.631135 1599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:03.631424 1599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:03.631587 1599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:03.638983 1599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:03.639073 1599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:03.639243 1599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:56:03.648582 1599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:44681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:45959
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:03.649889 1599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:03.651712 1599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:03.664908 1604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:03.664966 1607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:03.664937 1605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:03.665778 1599 server_base.cc:1034] running on GCE node
I20260929 15:56:03.666405 1599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:03.667439 1599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:03.668664 1599 hybrid_clock.cc:648] HybridClock initialized: now 1790697363668621 us; error 55 us; skew 500 ppm
I20260929 15:56:03.668992 1599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:56:03.671185 1599 webserver.cc:460] Webserver started at http://127.15.196.194:35735/ using document root <none> and password file <none>
I20260929 15:56:03.671945 1599 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:03.672076 1599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:03.672392 1599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:03.675266 1599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "31e8e0a923d84d7695358da8a5d30b9a"
format_stamp: "Formatted at 2026-09-29 15:56:03 on dist-test-slave-c51v"
server_key: "e121fe7e433e1c70009c0b5c867ddcbf"
server_key_iv: "a595a0a354443522d7e2061c3d483ee8"
server_key_version: "encryptionkey@0"
I20260929 15:56:03.676133 1599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "31e8e0a923d84d7695358da8a5d30b9a"
format_stamp: "Formatted at 2026-09-29 15:56:03 on dist-test-slave-c51v"
server_key: "e121fe7e433e1c70009c0b5c867ddcbf"
server_key_iv: "a595a0a354443522d7e2061c3d483ee8"
server_key_version: "encryptionkey@0"
I20260929 15:56:03.681330 1599 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:56:03.684684 1613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:03.685961 1599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 15:56:03.686136 1599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "31e8e0a923d84d7695358da8a5d30b9a"
format_stamp: "Formatted at 2026-09-29 15:56:03 on dist-test-slave-c51v"
server_key: "e121fe7e433e1c70009c0b5c867ddcbf"
server_key_iv: "a595a0a354443522d7e2061c3d483ee8"
server_key_version: "encryptionkey@0"
I20260929 15:56:03.686342 1599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:03.726990 1599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:03.727902 1599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:03.728212 1599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:03.729166 1599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:03.730600 1599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:03.730712 1599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:03.730836 1599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:03.730926 1599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:03.762872 1599 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:38819
I20260929 15:56:03.762959 1726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:38819 every 8 connection(s)
I20260929 15:56:03.764271 1599 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:56:03.770489 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 1599
I20260929 15:56:03.770696 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:56:03.771275 16147 external_mini_cluster.cc:1428] Setting key cb0bd4546914365a2ab62176ac57f695
I20260929 15:56:03.774044 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:45959
--builtin_ntp_servers=127.15.196.212:44681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 15:56:03.776796 1597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:03.766835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:36559 (local address 127.15.196.254:45959)
0929 15:56:03.767012 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:03.767016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:03.768381 (+ 1365us) server_negotiation.cc:407] Connection header received
0929 15:56:03.770103 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:03.770107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:03.770174 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:03.770258 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:03.771695 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:03.772417 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:03.773941 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:03.774444 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:03.774574 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:03.775998 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:03.776004 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:03.776015 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:03.776033 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:03.776064 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:03.776075 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:03.776166 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:03.776402 (+ 236us) server_negotiation.cc:299] Negotiation successful
0929 15:56:03.776496 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 15:56:03.777791 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:03.767236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:45959 (local address 127.15.196.194:36559)
0929 15:56:03.768239 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:03.768280 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:03.769857 (+ 1577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:03.770536 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:03.770548 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:03.770685 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:03.771529 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:03.771552 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:03.772584 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:03.772589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:03.773799 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:03.773813 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:03.774659 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:03.774666 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:03.774959 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:03.775679 (+ 720us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:56:03.775681 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:03.775720 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:56:03.775806 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:03.776178 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:03.776191 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:03.776202 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 15:56:03.776374 (+ 172us) client_negotiation.cc:240] Negotiation successful
0929 15:56:03.776760 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":307,"threads_started":1}
I20260929 15:56:03.779213 1727 heartbeater.cc:347] Connected to a master server at 127.15.196.254:45959
I20260929 15:56:03.779525 1727 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:03.780208 1727 heartbeater.cc:511] Master 127.15.196.254:45959 requested a full tablet report, sending...
I20260929 15:56:03.781615 1408 ts_manager.cc:194] Registered new tserver with Master: 31e8e0a923d84d7695358da8a5d30b9a (127.15.196.194:38819)
I20260929 15:56:03.782390 1408 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.196.194:36559
W20260929 15:56:04.094215 1730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:04.094534 1730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:04.094702 1730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:04.101815 1730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:04.101953 1730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:04.102134 1730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:56:04.111125 1730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:44681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:45959
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:04.112452 1730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:04.114235 1730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:04.127032 1737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:04.127032 1739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:04.127854 1735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:04.127884 1730 server_base.cc:1034] running on GCE node
I20260929 15:56:04.128515 1730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:04.129458 1730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:04.130688 1730 hybrid_clock.cc:648] HybridClock initialized: now 1790697364130643 us; error 57 us; skew 500 ppm
I20260929 15:56:04.131009 1730 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:56:04.133303 1730 webserver.cc:460] Webserver started at http://127.15.196.195:43499/ using document root <none> and password file <none>
I20260929 15:56:04.134008 1730 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:04.134140 1730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:04.134505 1730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:04.137516 1730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "e51cb77e77da49eb853d1bb89cd39d68"
format_stamp: "Formatted at 2026-09-29 15:56:04 on dist-test-slave-c51v"
server_key: "8a7dd4c3579c74aa385b0a14ae886b1c"
server_key_iv: "6e4c19a924a883d566fb306ce36b8de7"
server_key_version: "encryptionkey@0"
I20260929 15:56:04.138278 1730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "e51cb77e77da49eb853d1bb89cd39d68"
format_stamp: "Formatted at 2026-09-29 15:56:04 on dist-test-slave-c51v"
server_key: "8a7dd4c3579c74aa385b0a14ae886b1c"
server_key_iv: "6e4c19a924a883d566fb306ce36b8de7"
server_key_version: "encryptionkey@0"
I20260929 15:56:04.143790 1730 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 15:56:04.147854 1745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:04.149622 1730 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260929 15:56:04.149821 1730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "e51cb77e77da49eb853d1bb89cd39d68"
format_stamp: "Formatted at 2026-09-29 15:56:04 on dist-test-slave-c51v"
server_key: "8a7dd4c3579c74aa385b0a14ae886b1c"
server_key_iv: "6e4c19a924a883d566fb306ce36b8de7"
server_key_version: "encryptionkey@0"
I20260929 15:56:04.150053 1730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:04.164119 1730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:04.164877 1730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:04.165169 1730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:04.166059 1730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:04.167484 1730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:04.167608 1730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:04.167727 1730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:04.167819 1730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:04.200270 1730 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:44783
I20260929 15:56:04.200312 1858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:44783 every 8 connection(s)
I20260929 15:56:04.201776 1730 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:56:04.208452 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 1730
I20260929 15:56:04.208643 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:56:04.209093 16147 external_mini_cluster.cc:1428] Setting key a057fee97db65e801271203e84a24136
I20260929 15:56:04.214712 1597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.204422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:56275 (local address 127.15.196.254:45959)
0929 15:56:04.204606 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:04.204611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:04.206048 (+ 1437us) server_negotiation.cc:407] Connection header received
0929 15:56:04.207867 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:04.207873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:04.207947 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:04.208039 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:04.209204 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.210154 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.212286 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.212691 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.212787 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.214078 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:04.214082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:04.214085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:04.214102 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:04.214129 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:04.214140 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:04.214207 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:04.214404 (+ 197us) server_negotiation.cc:299] Negotiation successful
0929 15:56:04.214478 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 15:56:04.215773 1861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.204958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:45959 (local address 127.15.196.195:56275)
0929 15:56:04.205869 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:04.205909 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:04.207590 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:04.208239 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:04.208248 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:04.208339 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:04.209022 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:04.209040 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.210344 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.210349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:04.212147 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:04.212162 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.212880 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.212885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:04.213085 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.213800 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:56:04.213802 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:04.213844 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:56:04.213923 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:04.214221 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:04.214228 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:04.214234 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 15:56:04.214378 (+ 144us) client_negotiation.cc:240] Negotiation successful
0929 15:56:04.214670 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":285,"threads_started":1}
I20260929 15:56:04.217072 1859 heartbeater.cc:347] Connected to a master server at 127.15.196.254:45959
I20260929 15:56:04.217372 1859 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:04.218096 1859 heartbeater.cc:511] Master 127.15.196.254:45959 requested a full tablet report, sending...
I20260929 15:56:04.219460 1408 ts_manager.cc:194] Registered new tserver with Master: e51cb77e77da49eb853d1bb89cd39d68 (127.15.196.195:44783)
I20260929 15:56:04.220261 1408 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.196.195:56275
I20260929 15:56:04.224287 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:56:04.233832 1597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.228137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58568 (local address 127.15.196.254:45959)
0929 15:56:04.228349 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:04.228357 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:04.228773 (+ 416us) server_negotiation.cc:407] Connection header received
0929 15:56:04.228904 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:04.228908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:04.228979 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:04.229057 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:04.229514 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.230194 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.231662 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.232061 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.232151 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.233160 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:04.233165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:04.233168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:04.233185 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:04.233214 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:04.233225 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:04.233292 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:04.233425 (+ 133us) server_negotiation.cc:299] Negotiation successful
0929 15:56:04.233495 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 15:56:04.237478 1408 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:56:04.244529 1408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:56:04.292830 1874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.256348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:38819 (local address 127.0.0.1:46312)
0929 15:56:04.282025 (+ 25677us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:04.282089 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:04.282389 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:04.284456 (+ 2067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:04.284464 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:04.284495 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:04.284937 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:04.284949 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.287530 (+ 2581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.287535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:04.289857 (+ 2322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:04.289868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.291857 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.291866 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:04.292040 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.292093 (+ 53us) client_negotiation.cc:769] Sending connection context
0929 15:56:04.292201 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 15:56:04.292416 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25313,"mutex_wait_us":1,"thread_start_us":28231,"threads_started":1}
I20260929 15:56:04.293385 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.256813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:44783 (local address 127.0.0.1:58778)
0929 15:56:04.283813 (+ 27000us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:04.283831 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:04.283958 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:04.284759 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:04.284765 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:04.284816 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:04.285237 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:04.285247 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.286865 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.286872 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:04.289315 (+ 2443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:04.289332 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.291493 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.291498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:04.292865 (+ 1367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.292897 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 15:56:04.292977 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 15:56:04.293097 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26883,"thread_start_us":26766,"threads_started":1}
I20260929 15:56:04.293524 1876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.257629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60056 (local address 127.15.196.193:35933)
0929 15:56:04.281077 (+ 23448us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:04.281090 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:04.284498 (+ 3408us) server_negotiation.cc:407] Connection header received
0929 15:56:04.284801 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:04.284808 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:04.285030 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:04.285181 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:04.286037 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.287558 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.290259 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.291777 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.291943 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.292140 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:04.292927 (+ 787us) server_negotiation.cc:299] Negotiation successful
0929 15:56:04.293105 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23193,"thread_start_us":23459,"threads_started":1}
I20260929 15:56:04.293964 1871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.257118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:35933 (local address 127.0.0.1:60056)
0929 15:56:04.284387 (+ 27269us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:04.284403 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:04.284602 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:04.285440 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:04.285444 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:04.285467 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:04.285858 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:04.285874 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.287686 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.287692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:04.290070 (+ 2378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:04.290085 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.292271 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.292277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:04.292443 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.292473 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 15:56:04.292556 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 15:56:04.292676 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27128,"thread_start_us":25540,"threads_started":1}
I20260929 15:56:04.294129 1872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.256715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46312 (local address 127.15.196.194:38819)
0929 15:56:04.283726 (+ 27011us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:04.283734 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:04.283768 (+ 34us) server_negotiation.cc:407] Connection header received
0929 15:56:04.283887 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:04.283898 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:04.284197 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:04.284410 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:04.285674 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.287397 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.290029 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.291641 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.293368 (+ 1727us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.293574 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:04.293646 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 15:56:04.293817 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26838,"thread_start_us":26726,"threads_started":1}
I20260929 15:56:04.294255 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.257433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58778 (local address 127.15.196.195:44783)
0929 15:56:04.281613 (+ 24180us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:04.281623 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:04.283953 (+ 2330us) server_negotiation.cc:407] Connection header received
0929 15:56:04.284078 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:04.284083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:04.284260 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:04.284481 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:04.285452 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.286722 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.289534 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.290936 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.291129 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.291377 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:04.293724 (+ 2347us) server_negotiation.cc:299] Negotiation successful
0929 15:56:04.293906 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23985,"thread_start_us":24207,"threads_started":1}
I20260929 15:56:04.297662 1530 tablet_service.cc:1467] Processing CreateTablet for tablet e5ca12a4c469477d91d65a71e7cdff8b (DEFAULT_TABLE table=test-table [id=68c21335ca664be4bbf4c622d1146d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:04.297856 1661 tablet_service.cc:1467] Processing CreateTablet for tablet e5ca12a4c469477d91d65a71e7cdff8b (DEFAULT_TABLE table=test-table [id=68c21335ca664be4bbf4c622d1146d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:04.299106 1530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5ca12a4c469477d91d65a71e7cdff8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:04.299213 1661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5ca12a4c469477d91d65a71e7cdff8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:04.302404 1793 tablet_service.cc:1467] Processing CreateTablet for tablet e5ca12a4c469477d91d65a71e7cdff8b (DEFAULT_TABLE table=test-table [id=68c21335ca664be4bbf4c622d1146d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:04.304565 1793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5ca12a4c469477d91d65a71e7cdff8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:04.310626 1877 tablet_bootstrap.cc:492] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3: Bootstrap starting.
I20260929 15:56:04.313292 1878 tablet_bootstrap.cc:492] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a: Bootstrap starting.
I20260929 15:56:04.314551 1877 tablet_bootstrap.cc:654] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:04.315855 1877 log.cc:824] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:04.316180 1595 heartbeater.cc:503] Master 127.15.196.254:45959 was elected leader, sending a full tablet report...
I20260929 15:56:04.317026 1878 tablet_bootstrap.cc:654] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:04.318498 1878 log.cc:824] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:04.323508 1881 tablet_bootstrap.cc:492] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68: Bootstrap starting.
I20260929 15:56:04.325016 1878 tablet_bootstrap.cc:492] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a: No bootstrap required, opened a new log
I20260929 15:56:04.325313 1878 ts_tablet_manager.cc:1397] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.005s
I20260929 15:56:04.327229 1881 tablet_bootstrap.cc:654] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:04.327688 1877 tablet_bootstrap.cc:492] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3: No bootstrap required, opened a new log
I20260929 15:56:04.327952 1877 ts_tablet_manager.cc:1397] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.004s
I20260929 15:56:04.328408 1881 log.cc:824] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:04.331267 1881 tablet_bootstrap.cc:492] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68: No bootstrap required, opened a new log
I20260929 15:56:04.331598 1881 ts_tablet_manager.cc:1397] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.005s
I20260929 15:56:04.335481 1877 raft_consensus.cc:348] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e8e0a923d84d7695358da8a5d30b9a" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38819 } } peers { permanent_uuid: "e51cb77e77da49eb853d1bb89cd39d68" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44783 } } peers { permanent_uuid: "f3bd322ac37e4079b32ab3291a7947f3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35933 } }
I20260929 15:56:04.335915 1877 raft_consensus.cc:374] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:04.335623 1878 raft_consensus.cc:348] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e8e0a923d84d7695358da8a5d30b9a" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38819 } } peers { permanent_uuid: "e51cb77e77da49eb853d1bb89cd39d68" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44783 } } peers { permanent_uuid: "f3bd322ac37e4079b32ab3291a7947f3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35933 } }
I20260929 15:56:04.336007 1877 raft_consensus.cc:729] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3bd322ac37e4079b32ab3291a7947f3, State: Initialized, Role: FOLLOWER
I20260929 15:56:04.336067 1878 raft_consensus.cc:374] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:04.336153 1878 raft_consensus.cc:729] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e8e0a923d84d7695358da8a5d30b9a, State: Initialized, Role: FOLLOWER
I20260929 15:56:04.336997 1877 consensus_queue.cc:260] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e8e0a923d84d7695358da8a5d30b9a" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38819 } } peers { permanent_uuid: "e51cb77e77da49eb853d1bb89cd39d68" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44783 } } peers { permanent_uuid: "f3bd322ac37e4079b32ab3291a7947f3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35933 } }
I20260929 15:56:04.336956 1878 consensus_queue.cc:260] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e8e0a923d84d7695358da8a5d30b9a" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38819 } } peers { permanent_uuid: "e51cb77e77da49eb853d1bb89cd39d68" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44783 } } peers { permanent_uuid: "f3bd322ac37e4079b32ab3291a7947f3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35933 } }
I20260929 15:56:04.338201 1727 heartbeater.cc:503] Master 127.15.196.254:45959 was elected leader, sending a full tablet report...
I20260929 15:56:04.338790 1878 ts_tablet_manager.cc:1428] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260929 15:56:04.341979 1877 ts_tablet_manager.cc:1428] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260929 15:56:04.342983 1881 raft_consensus.cc:348] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e8e0a923d84d7695358da8a5d30b9a" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38819 } } peers { permanent_uuid: "e51cb77e77da49eb853d1bb89cd39d68" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44783 } } peers { permanent_uuid: "f3bd322ac37e4079b32ab3291a7947f3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35933 } }
I20260929 15:56:04.343442 1881 raft_consensus.cc:374] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:04.343564 1881 raft_consensus.cc:729] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51cb77e77da49eb853d1bb89cd39d68, State: Initialized, Role: FOLLOWER
I20260929 15:56:04.344194 1881 consensus_queue.cc:260] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e8e0a923d84d7695358da8a5d30b9a" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38819 } } peers { permanent_uuid: "e51cb77e77da49eb853d1bb89cd39d68" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44783 } } peers { permanent_uuid: "f3bd322ac37e4079b32ab3291a7947f3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35933 } }
I20260929 15:56:04.345130 1859 heartbeater.cc:503] Master 127.15.196.254:45959 was elected leader, sending a full tablet report...
I20260929 15:56:04.345551 1881 ts_tablet_manager.cc:1428] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68: Time spent starting tablet: real 0.014s user 0.007s sys 0.005s
I20260929 15:56:04.445309 1884 raft_consensus.cc:482] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:56:04.445619 1884 raft_consensus.cc:504] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e8e0a923d84d7695358da8a5d30b9a" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38819 } } peers { permanent_uuid: "e51cb77e77da49eb853d1bb89cd39d68" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44783 } } peers { permanent_uuid: "f3bd322ac37e4079b32ab3291a7947f3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35933 } }
I20260929 15:56:04.447459 1884 leader_election.cc:288] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31e8e0a923d84d7695358da8a5d30b9a (127.15.196.194:38819), e51cb77e77da49eb853d1bb89cd39d68 (127.15.196.195:44783)
W20260929 15:56:04.453600 1860 tablet.cc:2369] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:56:04.454394 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.448043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:43071 (local address 127.15.196.195:44783)
0929 15:56:04.448274 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:04.448299 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:04.449085 (+ 786us) server_negotiation.cc:407] Connection header received
0929 15:56:04.449287 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:04.449292 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:04.449350 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:04.449447 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:04.450049 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.450764 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.452615 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.453453 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.453633 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.453703 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:04.453996 (+ 293us) server_negotiation.cc:299] Negotiation successful
0929 15:56:04.454133 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260929 15:56:04.454586 1886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.447934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:44783 (local address 127.15.196.193:43071)
0929 15:56:04.448970 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:04.448984 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:04.449130 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:04.449459 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:04.449464 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:04.449485 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:04.449888 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:04.449898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.450906 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.450913 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:04.452425 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:04.452437 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.453634 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.453640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:04.453833 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.453867 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 15:56:04.453962 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 15:56:04.454090 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":927,"mutex_wait_us":3,"thread_start_us":424,"threads_started":1}
I20260929 15:56:04.454696 1872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.447861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:41315 (local address 127.15.196.194:38819)
0929 15:56:04.448083 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:04.448091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:04.448993 (+ 902us) server_negotiation.cc:407] Connection header received
0929 15:56:04.449230 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:04.449235 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:04.449295 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:04.449402 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:04.449981 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.450684 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.452635 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.453545 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.453699 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.453773 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:04.454308 (+ 535us) server_negotiation.cc:299] Negotiation successful
0929 15:56:04.454439 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 15:56:04.455406 1887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.447842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:38819 (local address 127.15.196.193:41315)
0929 15:56:04.448876 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:04.448892 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:04.449066 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:04.449423 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:04.449427 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:04.449449 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:04.449827 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:04.449840 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.450822 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.450829 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:04.452436 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:04.452447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.453928 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.453937 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:04.454090 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.454118 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 15:56:04.454873 (+ 755us) client_negotiation.cc:240] Negotiation successful
0929 15:56:04.455005 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":881,"mutex_wait_us":97,"thread_start_us":737,"threads_started":1}
I20260929 15:56:04.455765 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ca12a4c469477d91d65a71e7cdff8b" candidate_uuid: "f3bd322ac37e4079b32ab3291a7947f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51cb77e77da49eb853d1bb89cd39d68" is_pre_election: true
I20260929 15:56:04.456347 1814 raft_consensus.cc:2454] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3bd322ac37e4079b32ab3291a7947f3 in term 0.
I20260929 15:56:04.456458 1681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ca12a4c469477d91d65a71e7cdff8b" candidate_uuid: "f3bd322ac37e4079b32ab3291a7947f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e8e0a923d84d7695358da8a5d30b9a" is_pre_election: true
I20260929 15:56:04.456959 1681 raft_consensus.cc:2454] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3bd322ac37e4079b32ab3291a7947f3 in term 0.
I20260929 15:56:04.457042 1485 leader_election.cc:302] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51cb77e77da49eb853d1bb89cd39d68, f3bd322ac37e4079b32ab3291a7947f3; no voters:
I20260929 15:56:04.457454 1884 raft_consensus.cc:2781] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:56:04.457592 1884 raft_consensus.cc:482] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:56:04.457712 1884 raft_consensus.cc:3037] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:04.460035 1884 raft_consensus.cc:504] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e8e0a923d84d7695358da8a5d30b9a" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38819 } } peers { permanent_uuid: "e51cb77e77da49eb853d1bb89cd39d68" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44783 } } peers { permanent_uuid: "f3bd322ac37e4079b32ab3291a7947f3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35933 } }
I20260929 15:56:04.460668 1884 leader_election.cc:288] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [CANDIDATE]: Term 1 election: Requested vote from peers 31e8e0a923d84d7695358da8a5d30b9a (127.15.196.194:38819), e51cb77e77da49eb853d1bb89cd39d68 (127.15.196.195:44783)
I20260929 15:56:04.461061 1681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ca12a4c469477d91d65a71e7cdff8b" candidate_uuid: "f3bd322ac37e4079b32ab3291a7947f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e8e0a923d84d7695358da8a5d30b9a"
I20260929 15:56:04.461189 1814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5ca12a4c469477d91d65a71e7cdff8b" candidate_uuid: "f3bd322ac37e4079b32ab3291a7947f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51cb77e77da49eb853d1bb89cd39d68"
I20260929 15:56:04.461239 1681 raft_consensus.cc:3037] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:04.461395 1814 raft_consensus.cc:3037] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:04.463575 1681 raft_consensus.cc:2454] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3bd322ac37e4079b32ab3291a7947f3 in term 1.
I20260929 15:56:04.463948 1483 leader_election.cc:302] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31e8e0a923d84d7695358da8a5d30b9a, f3bd322ac37e4079b32ab3291a7947f3; no voters:
I20260929 15:56:04.464212 1884 raft_consensus.cc:2781] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:04.464203 1814 raft_consensus.cc:2454] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3bd322ac37e4079b32ab3291a7947f3 in term 1.
I20260929 15:56:04.464733 1884 raft_consensus.cc:686] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [term 1 LEADER]: Becoming Leader. State: Replica: f3bd322ac37e4079b32ab3291a7947f3, State: Running, Role: LEADER
I20260929 15:56:04.465312 1884 consensus_queue.cc:237] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e8e0a923d84d7695358da8a5d30b9a" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38819 } } peers { permanent_uuid: "e51cb77e77da49eb853d1bb89cd39d68" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44783 } } peers { permanent_uuid: "f3bd322ac37e4079b32ab3291a7947f3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35933 } }
I20260929 15:56:04.470148 1408 catalog_manager.cc:5495] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3bd322ac37e4079b32ab3291a7947f3 (127.15.196.193). New cstate: current_term: 1 leader_uuid: "f3bd322ac37e4079b32ab3291a7947f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e8e0a923d84d7695358da8a5d30b9a" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e51cb77e77da49eb853d1bb89cd39d68" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3bd322ac37e4079b32ab3291a7947f3" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 35933 } health_report { overall_health: HEALTHY } } }
I20260929 15:56:04.486917 1876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.481996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60070 (local address 127.15.196.193:35933)
0929 15:56:04.482170 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:04.482174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:04.482207 (+ 33us) server_negotiation.cc:407] Connection header received
0929 15:56:04.482400 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:04.482403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:04.482484 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:04.482600 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:04.482925 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.483680 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.484997 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.485401 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.485488 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.485880 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:04.486373 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:04.486482 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:04.486549 (+ 67us) server_negotiation.cc:299] Negotiation successful
0929 15:56:04.486607 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 15:56:04.494828 1681 raft_consensus.cc:1261] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a [term 1 FOLLOWER]: Refusing update from remote peer f3bd322ac37e4079b32ab3291a7947f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:56:04.494886 1814 raft_consensus.cc:1261] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68 [term 1 FOLLOWER]: Refusing update from remote peer f3bd322ac37e4079b32ab3291a7947f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:56:04.495826 1888 consensus_queue.cc:1035] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51cb77e77da49eb853d1bb89cd39d68" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 44783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:56:04.496169 1884 consensus_queue.cc:1035] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31e8e0a923d84d7695358da8a5d30b9a" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 38819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:56:04.507725 1894 mvcc.cc:205] Tried to move back new op lower bound from 7334696404961607680 to 7334696404854104064. Current Snapshot: MvccSnapshot[applied={T|T < 7334696404961607680}]
I20260929 15:56:04.520507 1408 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:56:04.520721 1408 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:56:04.524183 1408 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5695a979aba44ab39b3f26743c352b46: Sending DeleteTablet for 3 replicas of tablet e5ca12a4c469477d91d65a71e7cdff8b
I20260929 15:56:04.525323 1793 tablet_service.cc:1514] Processing DeleteTablet for tablet e5ca12a4c469477d91d65a71e7cdff8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:04 UTC) from {username='slave'} at 127.0.0.1:58778
I20260929 15:56:04.525331 1661 tablet_service.cc:1514] Processing DeleteTablet for tablet e5ca12a4c469477d91d65a71e7cdff8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:04 UTC) from {username='slave'} at 127.0.0.1:46312
I20260929 15:56:04.525678 1530 tablet_service.cc:1514] Processing DeleteTablet for tablet e5ca12a4c469477d91d65a71e7cdff8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:04 UTC) from {username='slave'} at 127.0.0.1:60056
I20260929 15:56:04.527541 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 1468
I20260929 15:56:04.540266 1901 tablet_replica.cc:330] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a: stopping tablet replica
I20260929 15:56:04.541412 1902 tablet_replica.cc:330] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68: stopping tablet replica
I20260929 15:56:04.541548 1901 raft_consensus.cc:2229] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:56:04.541879 1902 raft_consensus.cc:2229] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:56:04.542109 1903 tablet_replica.cc:330] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3: stopping tablet replica
I20260929 15:56:04.542115 1901 raft_consensus.cc:2258] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:04.542357 1902 raft_consensus.cc:2258] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:04.542642 1903 raft_consensus.cc:2229] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:56:04.543316 1903 raft_consensus.cc:2258] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:04.544394 1901 ts_tablet_manager.cc:1905] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:56:04.544406 1902 ts_tablet_manager.cc:1905] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:56:04.546221 1903 ts_tablet_manager.cc:1905] T e5ca12a4c469477d91d65a71e7cdff8b P f3bd322ac37e4079b32ab3291a7947f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:56:04.548775 1876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.533189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60072 (local address 127.15.196.193:35933)
0929 15:56:04.533619 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:04.533627 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:04.534096 (+ 469us) server_negotiation.cc:407] Connection header received
0929 15:56:04.534322 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:04.534329 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:04.534428 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:04.534536 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:04.541279 (+ 6743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.542678 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.544795 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.545497 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.545609 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.547044 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:04.547055 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:04.547074 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:04.547827 (+ 753us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:04.547883 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:04.547908 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:04.548014 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:04.548267 (+ 253us) server_negotiation.cc:299] Negotiation successful
0929 15:56:04.548395 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311}
I20260929 15:56:04.549945 1590 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:04.551805 1902 ts_tablet_manager.cc:1918] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:56:04.551929 1901 ts_tablet_manager.cc:1918] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:56:04.552021 1902 log.cc:1196] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/e5ca12a4c469477d91d65a71e7cdff8b
I20260929 15:56:04.552114 1901 log.cc:1196] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/e5ca12a4c469477d91d65a71e7cdff8b
I20260929 15:56:04.552597 1902 ts_tablet_manager.cc:1939] T e5ca12a4c469477d91d65a71e7cdff8b P e51cb77e77da49eb853d1bb89cd39d68: Deleting consensus metadata
I20260929 15:56:04.552731 1901 ts_tablet_manager.cc:1939] T e5ca12a4c469477d91d65a71e7cdff8b P 31e8e0a923d84d7695358da8a5d30b9a: Deleting consensus metadata
I20260929 15:56:04.555027 1394 catalog_manager.cc:4841] TS e51cb77e77da49eb853d1bb89cd39d68 (127.15.196.195:44783): tablet e5ca12a4c469477d91d65a71e7cdff8b (table test-table [id=68c21335ca664be4bbf4c622d1146d53]) successfully deleted
I20260929 15:56:04.555073 1392 catalog_manager.cc:4841] TS 31e8e0a923d84d7695358da8a5d30b9a (127.15.196.194:38819): tablet e5ca12a4c469477d91d65a71e7cdff8b (table test-table [id=68c21335ca664be4bbf4c622d1146d53]) successfully deleted
I20260929 15:56:04.648991 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 1468
W20260929 15:56:04.668059 1393 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.196.193:35933 (error 108)
I20260929 15:56:04.668133 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 1599
I20260929 15:56:04.670063 1871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.669576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:35933 (local address 127.0.0.1:60088)
0929 15:56:04.669779 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:04.669908 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:35933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260929 15:56:04.670331 1393 catalog_manager.cc:4584] TS f3bd322ac37e4079b32ab3291a7947f3 (127.15.196.193:35933): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e5ca12a4c469477d91d65a71e7cdff8b: Network error: Client connection negotiation failed: client connection to 127.15.196.193:35933: connect: Connection refused (error 111)
I20260929 15:56:04.678517 1872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.669066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46316 (local address 127.15.196.194:38819)
0929 15:56:04.669266 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:04.669272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:04.669304 (+ 32us) server_negotiation.cc:407] Connection header received
0929 15:56:04.669497 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:04.669508 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:04.669621 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:04.669733 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:04.670150 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.671080 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.672443 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.672850 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.672989 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.677078 (+ 4089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:04.677088 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:04.677103 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:04.677705 (+ 602us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:04.677756 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:04.677777 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:04.677908 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:04.678043 (+ 135us) server_negotiation.cc:299] Negotiation successful
0929 15:56:04.678166 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 15:56:04.679225 1722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:04.724001 1871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.723423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:35933 (local address 127.0.0.1:60096)
0929 15:56:04.723648 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:04.723784 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:35933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260929 15:56:04.725083 1871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.724686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:35933 (local address 127.0.0.1:60104)
0929 15:56:04.724839 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:04.724925 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:35933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 15:56:04.769045 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 1599
I20260929 15:56:04.785813 1871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.785231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:35933 (local address 127.0.0.1:60112)
0929 15:56:04.785537 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:04.785636 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:35933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260929 15:56:04.786903 1871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.786544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:35933 (local address 127.0.0.1:60124)
0929 15:56:04.786674 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:04.786739 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:35933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260929 15:56:04.787323 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 1730
I20260929 15:56:04.793543 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.787995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58794 (local address 127.15.196.195:44783)
0929 15:56:04.788161 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:04.788165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:04.788182 (+ 17us) server_negotiation.cc:407] Connection header received
0929 15:56:04.788307 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:04.788310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:04.788394 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:04.788475 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:04.788839 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.789587 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.790791 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:04.791158 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:04.791249 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:04.792357 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:04.792365 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:04.792379 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:04.792941 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:04.792987 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:04.793018 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:04.793100 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:04.793256 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 15:56:04.793334 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 15:56:04.794306 1854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:04.867738 1871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.867134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:35933 (local address 127.0.0.1:60132)
0929 15:56:04.867427 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:04.867554 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:35933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260929 15:56:04.868860 1871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:04.868429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:35933 (local address 127.0.0.1:60148)
0929 15:56:04.868642 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:04.868727 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:35933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260929 15:56:04.884255 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 1730
I20260929 15:56:04.900957 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 1377
I20260929 15:56:04.902034 1439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:05.016942 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 1377
2026-09-29T15:56:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2826 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:05 dist-test-slave-c51v krb5kdc[1912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:05 dist-test-slave-c51v krb5kdc[1912](info): set up 2 sockets
Sep 29 15:56:05 dist-test-slave-c51v krb5kdc[1912](info): commencing operation
krb5kdc: starting...
W20260929 15:56:06.117545 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.007s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:06 dist-test-slave-c51v krb5kdc[1912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:56:06.373900 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:39265
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:39265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:06.703167 1925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:06.703545 1925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:06.703722 1925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:06.711071 1925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:06.711174 1925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:06.711309 1925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:06.711413 1925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:06.711473 1925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:06.721330 1925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:39265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:39265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:06.722654 1925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:06.724044 1925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:06.735807 1933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:06.735908 1931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:06.735862 1930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:06.736938 1925 server_base.cc:1034] running on GCE node
I20260929 15:56:06.737643 1925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:06.739002 1925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:06.740203 1925 hybrid_clock.cc:648] HybridClock initialized: now 1790697366740167 us; error 29 us; skew 500 ppm
I20260929 15:56:06.740561 1925 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:06 dist-test-slave-c51v krb5kdc[1912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697366, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:06.743602 1925 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:56:06.745030 1925 webserver.cc:460] Webserver started at http://127.15.196.254:37711/ using document root <none> and password file <none>
I20260929 15:56:06.745707 1925 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:06.745860 1925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:06.746187 1925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:06.749245 1925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "c3b3fea79dc048e2bb518b0b3593ef92"
format_stamp: "Formatted at 2026-09-29 15:56:06 on dist-test-slave-c51v"
server_key: "032aec85481e68d2195a161a4e5074ba"
server_key_iv: "56442e228f50ac59a6627b9c08fb2b9d"
server_key_version: "encryptionkey@0"
I20260929 15:56:06.750095 1925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "c3b3fea79dc048e2bb518b0b3593ef92"
format_stamp: "Formatted at 2026-09-29 15:56:06 on dist-test-slave-c51v"
server_key: "032aec85481e68d2195a161a4e5074ba"
server_key_iv: "56442e228f50ac59a6627b9c08fb2b9d"
server_key_version: "encryptionkey@0"
I20260929 15:56:06.755206 1925 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 15:56:06.758625 1940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:06.760056 1925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 15:56:06.760241 1925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "c3b3fea79dc048e2bb518b0b3593ef92"
format_stamp: "Formatted at 2026-09-29 15:56:06 on dist-test-slave-c51v"
server_key: "032aec85481e68d2195a161a4e5074ba"
server_key_iv: "56442e228f50ac59a6627b9c08fb2b9d"
server_key_version: "encryptionkey@0"
I20260929 15:56:06.760449 1925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:06.798404 1925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:06.802814 1925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:06.803145 1925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:06.822014 1925 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:39265
I20260929 15:56:06.822024 1991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:39265 every 8 connection(s)
I20260929 15:56:06.823465 1925 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:06.827136 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 1925
I20260929 15:56:06.827360 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:06.827391 1992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:06.827940 16147 external_mini_cluster.cc:1428] Setting key 2900c6af623442f833703c30647a5e90
I20260929 15:56:06.842479 1992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92: Bootstrap starting.
Sep 29 15:56:06 dist-test-slave-c51v krb5kdc[1912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:56:06.845738 1992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:06.846927 1992 log.cc:824] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:06.849937 1992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92: No bootstrap required, opened a new log
I20260929 15:56:06.852312 1995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:06.829821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46042 (local address 127.15.196.254:39265)
0929 15:56:06.830843 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:06.830862 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:06.831100 (+ 238us) server_negotiation.cc:407] Connection header received
0929 15:56:06.832964 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:06.832985 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:06.833383 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:06.833720 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:06.835520 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:06.836821 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:06.838666 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:06.839491 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:06.839644 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:06.844636 (+ 4992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:06.844644 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:06.844660 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:06.844704 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:06.848078 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:06.848787 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:06.848799 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:06.848805 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:06.848965 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:06.849382 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:06.849392 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:06.849394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:06.849953 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:06.850249 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:06.850812 (+ 563us) server_negotiation.cc:299] Negotiation successful
0929 15:56:06.851242 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":410,"threads_started":1}
I20260929 15:56:06.858528 1992 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b3fea79dc048e2bb518b0b3593ef92" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 39265 } }
I20260929 15:56:06.858916 1992 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:06.859020 1992 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3fea79dc048e2bb518b0b3593ef92, State: Initialized, Role: FOLLOWER
I20260929 15:56:06.859624 1992 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b3fea79dc048e2bb518b0b3593ef92" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 39265 } }
I20260929 15:56:06.859879 1992 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:06.860016 1992 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:06.860186 1992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:06.862326 1992 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b3fea79dc048e2bb518b0b3593ef92" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 39265 } }
I20260929 15:56:06.862840 1992 leader_election.cc:302] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3b3fea79dc048e2bb518b0b3593ef92; no voters:
I20260929 15:56:06.863483 1992 leader_election.cc:288] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:06.863610 1997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:06.864362 1997 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [term 1 LEADER]: Becoming Leader. State: Replica: c3b3fea79dc048e2bb518b0b3593ef92, State: Running, Role: LEADER
I20260929 15:56:06.865028 1997 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b3fea79dc048e2bb518b0b3593ef92" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 39265 } }
I20260929 15:56:06.865931 1992 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:06.868132 1999 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3b3fea79dc048e2bb518b0b3593ef92. Latest consensus state: current_term: 1 leader_uuid: "c3b3fea79dc048e2bb518b0b3593ef92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b3fea79dc048e2bb518b0b3593ef92" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 39265 } } }
I20260929 15:56:06.868108 1998 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3b3fea79dc048e2bb518b0b3593ef92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b3fea79dc048e2bb518b0b3593ef92" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 39265 } } }
I20260929 15:56:06.868533 1998 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:06.868533 1999 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:06.869339 2003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:06.874573 2003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:06.885571 2003 catalog_manager.cc:1347] Generated new cluster ID: ee403ff0e9b84c89a9bd27eb4635451c
I20260929 15:56:06.885705 2003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:06.905138 2003 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:06.906490 2003 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:06.915678 2003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92: Generated new TSK 0
I20260929 15:56:06.916509 2003 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:56:06.984305 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:39265
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:07.329808 2019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:07.330060 2019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:07.330159 2019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:07.337188 2019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:07.337289 2019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:07.337393 2019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:56:07.346724 2019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:39265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:07.348027 2019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:07.349397 2019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:07.362957 2025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:07.363150 2024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:07.363098 2027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:07.364218 2019 server_base.cc:1034] running on GCE node
I20260929 15:56:07.364888 2019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:07.365899 2019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:07.367096 2019 hybrid_clock.cc:648] HybridClock initialized: now 1790697367367031 us; error 65 us; skew 500 ppm
I20260929 15:56:07.367533 2019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:07 dist-test-slave-c51v krb5kdc[1912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697367, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:07.370692 2019 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:56:07.372210 2019 webserver.cc:460] Webserver started at http://127.15.196.193:34345/ using document root <none> and password file <none>
I20260929 15:56:07.372928 2019 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:07.373075 2019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:07.373391 2019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:07.376350 2019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "14791172465d4e268abedc032776781b"
format_stamp: "Formatted at 2026-09-29 15:56:07 on dist-test-slave-c51v"
server_key: "c613b33281da9a0680b1cd6fab3eb81c"
server_key_iv: "1f37975376ac3ec9fa3fb5aa793bdd64"
server_key_version: "encryptionkey@0"
I20260929 15:56:07.377156 2019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "14791172465d4e268abedc032776781b"
format_stamp: "Formatted at 2026-09-29 15:56:07 on dist-test-slave-c51v"
server_key: "c613b33281da9a0680b1cd6fab3eb81c"
server_key_iv: "1f37975376ac3ec9fa3fb5aa793bdd64"
server_key_version: "encryptionkey@0"
I20260929 15:56:07.382496 2019 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 15:56:07.385902 2034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:07.387152 2019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 15:56:07.387364 2019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "14791172465d4e268abedc032776781b"
format_stamp: "Formatted at 2026-09-29 15:56:07 on dist-test-slave-c51v"
server_key: "c613b33281da9a0680b1cd6fab3eb81c"
server_key_iv: "1f37975376ac3ec9fa3fb5aa793bdd64"
server_key_version: "encryptionkey@0"
I20260929 15:56:07.387563 2019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:07.402570 2019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:07.406464 2019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:07.406754 2019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:07.407728 2019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:07.409015 2019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:07.409116 2019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:07.409261 2019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:07.409339 2019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:07.441923 2019 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:38503
I20260929 15:56:07.441965 2146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:38503 every 8 connection(s)
I20260929 15:56:07.443202 2019 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:56:07.448257 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 2019
I20260929 15:56:07.448449 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:56:07.448906 16147 external_mini_cluster.cc:1428] Setting key ec399918abf0b02caa9be74581149236
Sep 29 15:56:07 dist-test-slave-c51v krb5kdc[1912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697367, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:56:07.463430 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:07.445743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:36655 (local address 127.15.196.254:39265)
0929 15:56:07.446149 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:07.446154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:07.447190 (+ 1036us) server_negotiation.cc:407] Connection header received
0929 15:56:07.449226 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:07.449232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:07.449325 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:07.449456 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:07.450976 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:07.451833 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:07.453374 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:07.453799 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:07.453935 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:07.457650 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:07.457655 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:07.457659 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:07.457691 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:07.460428 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:07.461132 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:07.461139 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:07.461142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:07.461268 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:07.461679 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:07.461686 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:07.461689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:07.461950 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:07.462127 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:07.462821 (+ 694us) server_negotiation.cc:299] Negotiation successful
0929 15:56:07.463042 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":193,"threads_started":1}
I20260929 15:56:07.464563 2150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:07.446329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:39265 (local address 127.15.196.193:36655)
0929 15:56:07.447018 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:07.447062 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:07.448906 (+ 1844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:07.449717 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:07.449726 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:07.450249 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:07.450808 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:07.450825 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:07.452012 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:07.452016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:07.453220 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:07.453242 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:07.454026 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:07.454034 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:07.454278 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:07.454985 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:07.455006 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:07.457425 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:07.460569 (+ 3144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:07.460578 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:07.460595 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:07.461008 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:07.461378 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:07.461384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:07.461387 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:07.461571 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:07.462101 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:07.462108 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:07.462454 (+ 346us) client_negotiation.cc:769] Sending connection context
0929 15:56:07.462715 (+ 261us) client_negotiation.cc:240] Negotiation successful
0929 15:56:07.463241 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":264,"threads_started":1}
I20260929 15:56:07.466248 2147 heartbeater.cc:347] Connected to a master server at 127.15.196.254:39265
I20260929 15:56:07.466583 2147 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:07.467515 2147 heartbeater.cc:511] Master 127.15.196.254:39265 requested a full tablet report, sending...
I20260929 15:56:07.469980 1957 ts_manager.cc:194] Registered new tserver with Master: 14791172465d4e268abedc032776781b (127.15.196.193:38503)
I20260929 15:56:07.472273 1957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:36655
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:56:07.506047 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:39265
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:07.843438 2153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:07.843708 2153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:07.843812 2153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:07.851035 2153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:07.851147 2153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:07.851260 2153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:56:07.861280 2153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:39265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:07.862563 2153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:07.863967 2153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:07.877242 2158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:07.877269 2159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:07.877315 2161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:07.878363 2153 server_base.cc:1034] running on GCE node
I20260929 15:56:07.879012 2153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:07.880043 2153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:07.881301 2153 hybrid_clock.cc:648] HybridClock initialized: now 1790697367881217 us; error 43 us; skew 500 ppm
I20260929 15:56:07.881683 2153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:07 dist-test-slave-c51v krb5kdc[1912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697367, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:07.885205 2153 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:56:07.886619 2153 webserver.cc:460] Webserver started at http://127.15.196.194:40033/ using document root <none> and password file <none>
I20260929 15:56:07.887372 2153 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:07.887508 2153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:07.887836 2153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:07.890828 2153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "2885bccd67924828bdf6f7034c144870"
format_stamp: "Formatted at 2026-09-29 15:56:07 on dist-test-slave-c51v"
server_key: "e9b416b0c71d23f1e79fff83ca8d2d8b"
server_key_iv: "bc672a945d5921f2a8175976821f8237"
server_key_version: "encryptionkey@0"
I20260929 15:56:07.891628 2153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "2885bccd67924828bdf6f7034c144870"
format_stamp: "Formatted at 2026-09-29 15:56:07 on dist-test-slave-c51v"
server_key: "e9b416b0c71d23f1e79fff83ca8d2d8b"
server_key_iv: "bc672a945d5921f2a8175976821f8237"
server_key_version: "encryptionkey@0"
I20260929 15:56:07.896626 2153 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260929 15:56:07.899993 2168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:07.901280 2153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:56:07.901458 2153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "2885bccd67924828bdf6f7034c144870"
format_stamp: "Formatted at 2026-09-29 15:56:07 on dist-test-slave-c51v"
server_key: "e9b416b0c71d23f1e79fff83ca8d2d8b"
server_key_iv: "bc672a945d5921f2a8175976821f8237"
server_key_version: "encryptionkey@0"
I20260929 15:56:07.901640 2153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:07.923522 2153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:07.927383 2153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:07.927695 2153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:07.928620 2153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:07.929916 2153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:07.930025 2153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:07.930173 2153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:07.930246 2153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:07.964681 2153 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:37937
I20260929 15:56:07.964702 2281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:37937 every 8 connection(s)
I20260929 15:56:07.966316 2153 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:56:07.971158 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 2153
I20260929 15:56:07.971387 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:56:07.971891 16147 external_mini_cluster.cc:1428] Setting key c39e3c9aed3709dbcdb5d5a9e0a707a1
Sep 29 15:56:07 dist-test-slave-c51v krb5kdc[1912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697367, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:56:07.986406 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:07.969066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:42625 (local address 127.15.196.254:39265)
0929 15:56:07.969531 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:07.969537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:07.970587 (+ 1050us) server_negotiation.cc:407] Connection header received
0929 15:56:07.972568 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:07.972572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:07.972650 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:07.972798 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:07.974413 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:07.975163 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:07.976712 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:07.977160 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:07.977299 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:07.980652 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:07.980657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:07.980660 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:07.980693 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:07.983352 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:07.984055 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:07.984060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:07.984065 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:07.984198 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:07.984638 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:07.984643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:07.984644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:07.984908 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:07.985062 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:07.985772 (+ 710us) server_negotiation.cc:299] Negotiation successful
0929 15:56:07.986014 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":215,"threads_started":1}
I20260929 15:56:07.987442 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:07.969717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:39265 (local address 127.15.196.194:42625)
0929 15:56:07.970435 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:07.970472 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:07.972276 (+ 1804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:07.973045 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:07.973053 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:07.973556 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:07.974241 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:07.974259 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:07.975343 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:07.975352 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:07.976559 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:07.976571 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:07.977397 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:07.977404 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:07.977628 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:07.978307 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:07.978333 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:07.980427 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:07.983502 (+ 3075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:07.983511 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:07.983526 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:07.983929 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:07.984315 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:07.984319 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:07.984323 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:07.984506 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:07.985031 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:07.985038 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:07.985396 (+ 358us) client_negotiation.cc:769] Sending connection context
0929 15:56:07.985669 (+ 273us) client_negotiation.cc:240] Negotiation successful
0929 15:56:07.986205 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":287,"threads_started":1}
I20260929 15:56:07.988999 2282 heartbeater.cc:347] Connected to a master server at 127.15.196.254:39265
I20260929 15:56:07.989362 2282 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:07.990329 2282 heartbeater.cc:511] Master 127.15.196.254:39265 requested a full tablet report, sending...
I20260929 15:56:07.992205 1957 ts_manager.cc:194] Registered new tserver with Master: 2885bccd67924828bdf6f7034c144870 (127.15.196.194:37937)
I20260929 15:56:07.993558 1957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:42625
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:56:08.030985 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:39265
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:08.366416 2288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:08.366748 2288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:08.366973 2288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:08.374718 2288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:08.374863 2288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:08.375073 2288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:56:08.385294 2288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:39265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:08.386663 2288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:08.388064 2288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:08.406628 2294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:08.407223 2293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:08.407622 2296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:08.408037 2288 server_base.cc:1034] running on GCE node
I20260929 15:56:08.408694 2288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:08.409664 2288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:08.410902 2288 hybrid_clock.cc:648] HybridClock initialized: now 1790697368410860 us; error 53 us; skew 500 ppm
I20260929 15:56:08.411218 2288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:08 dist-test-slave-c51v krb5kdc[1912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697368, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:08.414296 2288 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:56:08.415848 2288 webserver.cc:460] Webserver started at http://127.15.196.195:43889/ using document root <none> and password file <none>
I20260929 15:56:08.416527 2288 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:08.416672 2288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:08.416992 2288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:08.419921 2288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "a5226fcd62c2498fb7882d1e9ceffae6"
format_stamp: "Formatted at 2026-09-29 15:56:08 on dist-test-slave-c51v"
server_key: "9421c135f5f429e163cab4aa51c3a324"
server_key_iv: "945d6a25cfbf619b710183b74d74334b"
server_key_version: "encryptionkey@0"
I20260929 15:56:08.420681 2288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "a5226fcd62c2498fb7882d1e9ceffae6"
format_stamp: "Formatted at 2026-09-29 15:56:08 on dist-test-slave-c51v"
server_key: "9421c135f5f429e163cab4aa51c3a324"
server_key_iv: "945d6a25cfbf619b710183b74d74334b"
server_key_version: "encryptionkey@0"
I20260929 15:56:08.425664 2288 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260929 15:56:08.429050 2303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:08.430338 2288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 15:56:08.430506 2288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "a5226fcd62c2498fb7882d1e9ceffae6"
format_stamp: "Formatted at 2026-09-29 15:56:08 on dist-test-slave-c51v"
server_key: "9421c135f5f429e163cab4aa51c3a324"
server_key_iv: "945d6a25cfbf619b710183b74d74334b"
server_key_version: "encryptionkey@0"
I20260929 15:56:08.430711 2288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:08.452752 2288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:08.456694 2288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:08.457013 2288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:08.457938 2288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:08.459227 2288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:08.459349 2288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:08.459497 2288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:08.459570 2288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:08.475639 2147 heartbeater.cc:503] Master 127.15.196.254:39265 was elected leader, sending a full tablet report...
I20260929 15:56:08.493144 2288 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:39183
I20260929 15:56:08.493183 2416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:39183 every 8 connection(s)
I20260929 15:56:08.494436 2288 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:56:08.496924 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 2288
I20260929 15:56:08.497126 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:56:08.497637 16147 external_mini_cluster.cc:1428] Setting key be0beb1fdfde03cb49e09e807be9890e
Sep 29 15:56:08 dist-test-slave-c51v krb5kdc[1912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697368, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:56:08.514864 2419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:08.496996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:37757 (local address 127.15.196.254:39265)
0929 15:56:08.497470 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:08.497477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:08.499011 (+ 1534us) server_negotiation.cc:407] Connection header received
0929 15:56:08.500768 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:08.500773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:08.500878 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:08.500992 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:08.502464 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.503232 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.504782 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.505233 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.505373 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:08.508847 (+ 3474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:08.508852 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:08.508855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:08.508888 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:08.511585 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:08.512364 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:08.512368 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:08.512370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:08.512530 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:08.513020 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:08.513028 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:08.513032 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:08.513309 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:08.513474 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:08.514249 (+ 775us) server_negotiation.cc:299] Negotiation successful
0929 15:56:08.514474 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":217,"threads_started":1}
I20260929 15:56:08.515900 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:08.497641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:39265 (local address 127.15.196.195:37757)
0929 15:56:08.498491 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:08.498538 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:08.500521 (+ 1983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:08.501226 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:08.501234 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:08.501739 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:08.502302 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:08.502319 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.503416 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.503421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:08.504629 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:08.504645 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.505458 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.505465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:08.505698 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:08.506445 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:08.506468 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:08.508647 (+ 2179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:08.511738 (+ 3091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:08.511749 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:08.511767 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:08.512239 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:08.512642 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:08.512645 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:08.512648 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:08.512910 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:08.513437 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:08.513445 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:08.513872 (+ 427us) client_negotiation.cc:769] Sending connection context
0929 15:56:08.514155 (+ 283us) client_negotiation.cc:240] Negotiation successful
0929 15:56:08.514703 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":323,"threads_started":1}
I20260929 15:56:08.517187 2417 heartbeater.cc:347] Connected to a master server at 127.15.196.254:39265
I20260929 15:56:08.517463 2417 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:08.518112 2417 heartbeater.cc:511] Master 127.15.196.254:39265 requested a full tablet report, sending...
I20260929 15:56:08.519519 1957 ts_manager.cc:194] Registered new tserver with Master: a5226fcd62c2498fb7882d1e9ceffae6 (127.15.196.195:39183)
I20260929 15:56:08.520323 1957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:37757
I20260929 15:56:08.525067 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:56:08.542092 2419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:08.529103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46050 (local address 127.15.196.254:39265)
0929 15:56:08.529261 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:08.529265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:08.529771 (+ 506us) server_negotiation.cc:407] Connection header received
0929 15:56:08.529907 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:08.529910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:08.529968 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:08.530047 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:08.531444 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.532152 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.533597 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.534176 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.534282 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:08.536980 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:08.536985 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:08.536988 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:08.537020 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:08.539272 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:08.540087 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:08.540093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:08.540096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:08.540217 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:08.540656 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:08.540661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:08.540664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:08.540973 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:08.541242 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:08.541608 (+ 366us) server_negotiation.cc:299] Negotiation successful
0929 15:56:08.541794 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 15:56:08.545784 1957 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:56:08.553083 1957 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:56:08.578565 2431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:08.565285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:39183 (local address 127.0.0.1:52016)
0929 15:56:08.566940 (+ 1655us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:08.566997 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:08.567200 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:08.568715 (+ 1515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:08.568723 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:08.568752 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:08.569223 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:08.569238 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.571243 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.571252 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:08.574438 (+ 3186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:08.574460 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.577074 (+ 2614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.577083 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:08.577426 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:08.577513 (+ 87us) client_negotiation.cc:769] Sending connection context
0929 15:56:08.577652 (+ 139us) client_negotiation.cc:240] Negotiation successful
0929 15:56:08.577917 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1327,"mutex_wait_us":1,"thread_start_us":3838,"threads_started":1}
I20260929 15:56:08.580358 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:08.565402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52016 (local address 127.15.196.195:39183)
0929 15:56:08.568227 (+ 2825us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:08.568232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:08.568258 (+ 26us) server_negotiation.cc:407] Connection header received
0929 15:56:08.568337 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:08.568342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:08.568517 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:08.568709 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:08.569473 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.571095 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.574733 (+ 3638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.576765 (+ 2032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.579517 (+ 2752us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:08.579744 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:08.579833 (+ 89us) server_negotiation.cc:299] Negotiation successful
0929 15:56:08.580014 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2701,"thread_start_us":2581,"threads_started":1}
I20260929 15:56:08.582325 2430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:08.565394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:37937 (local address 127.0.0.1:48266)
0929 15:56:08.569434 (+ 4040us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:08.569452 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:08.569644 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:08.570489 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:08.570494 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:08.570518 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:08.570845 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:08.570855 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.574905 (+ 4050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.574914 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:08.579183 (+ 4269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:08.579201 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.581475 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.581481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:08.581652 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:08.581687 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 15:56:08.581789 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 15:56:08.581959 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3909,"thread_start_us":372,"threads_started":1}
I20260929 15:56:08.582360 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:08.565899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48266 (local address 127.15.196.194:37937)
0929 15:56:08.567839 (+ 1940us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:08.567845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:08.569537 (+ 1692us) server_negotiation.cc:407] Connection header received
0929 15:56:08.569841 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:08.569852 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:08.570083 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:08.570229 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:08.571036 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.574681 (+ 3645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.579400 (+ 4719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.580999 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.581132 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:08.581300 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:08.581856 (+ 556us) server_negotiation.cc:299] Negotiation successful
0929 15:56:08.582023 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1784,"thread_start_us":1659,"threads_started":1}
I20260929 15:56:08.584576 2352 tablet_service.cc:1467] Processing CreateTablet for tablet d95661bcfe8b4320abde518b913d1efe (DEFAULT_TABLE table=test-table [id=9377f8cf56fd43e193c9fa0384e535cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:08.585711 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:08.569890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49074 (local address 127.15.196.193:38503)
0929 15:56:08.578911 (+ 9021us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:08.578920 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:08.578950 (+ 30us) server_negotiation.cc:407] Connection header received
0929 15:56:08.579146 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:08.579157 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:08.579539 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:08.579796 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:08.580178 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.581716 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.583857 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.584834 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.584990 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:08.585098 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:08.585369 (+ 271us) server_negotiation.cc:299] Negotiation successful
0929 15:56:08.585487 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8827,"thread_start_us":8670,"threads_started":1}
I20260929 15:56:08.585769 2431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:08.571101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:38503 (local address 127.0.0.1:49074)
0929 15:56:08.578813 (+ 7712us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:08.578833 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:08.578941 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:08.579810 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:08.579815 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:08.579851 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:08.580006 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:08.580016 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.581896 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.581901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:08.583654 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:08.583669 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.584999 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.585006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:08.585178 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:08.585216 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 15:56:08.585316 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 15:56:08.585462 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7614,"thread_start_us":8210,"threads_started":1}
I20260929 15:56:08.586109 2352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d95661bcfe8b4320abde518b913d1efe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:08.591032 2217 tablet_service.cc:1467] Processing CreateTablet for tablet d95661bcfe8b4320abde518b913d1efe (DEFAULT_TABLE table=test-table [id=9377f8cf56fd43e193c9fa0384e535cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:08.591144 2082 tablet_service.cc:1467] Processing CreateTablet for tablet d95661bcfe8b4320abde518b913d1efe (DEFAULT_TABLE table=test-table [id=9377f8cf56fd43e193c9fa0384e535cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:08.592371 2217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d95661bcfe8b4320abde518b913d1efe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:08.592588 2082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d95661bcfe8b4320abde518b913d1efe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:08.600790 2436 tablet_bootstrap.cc:492] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6: Bootstrap starting.
I20260929 15:56:08.604478 2436 tablet_bootstrap.cc:654] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:08.604599 2437 tablet_bootstrap.cc:492] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870: Bootstrap starting.
I20260929 15:56:08.606653 2436 log.cc:824] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:08.608834 2437 tablet_bootstrap.cc:654] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:08.609612 2436 tablet_bootstrap.cc:492] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6: No bootstrap required, opened a new log
I20260929 15:56:08.609876 2436 ts_tablet_manager.cc:1397] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260929 15:56:08.609854 2437 log.cc:824] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:08.612391 2437 tablet_bootstrap.cc:492] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870: No bootstrap required, opened a new log
I20260929 15:56:08.612530 2438 tablet_bootstrap.cc:492] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b: Bootstrap starting.
I20260929 15:56:08.612671 2437 ts_tablet_manager.cc:1397] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260929 15:56:08.616194 2438 tablet_bootstrap.cc:654] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:08.617936 2438 log.cc:824] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:08.619342 2436 raft_consensus.cc:348] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5226fcd62c2498fb7882d1e9ceffae6" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39183 } } peers { permanent_uuid: "2885bccd67924828bdf6f7034c144870" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37937 } } peers { permanent_uuid: "14791172465d4e268abedc032776781b" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38503 } }
I20260929 15:56:08.619804 2436 raft_consensus.cc:374] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:08.619903 2436 raft_consensus.cc:729] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5226fcd62c2498fb7882d1e9ceffae6, State: Initialized, Role: FOLLOWER
I20260929 15:56:08.620541 2436 consensus_queue.cc:260] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5226fcd62c2498fb7882d1e9ceffae6" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39183 } } peers { permanent_uuid: "2885bccd67924828bdf6f7034c144870" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37937 } } peers { permanent_uuid: "14791172465d4e268abedc032776781b" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38503 } }
I20260929 15:56:08.620735 2437 raft_consensus.cc:348] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5226fcd62c2498fb7882d1e9ceffae6" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39183 } } peers { permanent_uuid: "2885bccd67924828bdf6f7034c144870" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37937 } } peers { permanent_uuid: "14791172465d4e268abedc032776781b" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38503 } }
I20260929 15:56:08.621183 2437 raft_consensus.cc:374] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:08.621276 2438 tablet_bootstrap.cc:492] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b: No bootstrap required, opened a new log
I20260929 15:56:08.621292 2437 raft_consensus.cc:729] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2885bccd67924828bdf6f7034c144870, State: Initialized, Role: FOLLOWER
I20260929 15:56:08.621551 2438 ts_tablet_manager.cc:1397] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260929 15:56:08.621901 2417 heartbeater.cc:503] Master 127.15.196.254:39265 was elected leader, sending a full tablet report...
I20260929 15:56:08.621893 2437 consensus_queue.cc:260] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5226fcd62c2498fb7882d1e9ceffae6" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39183 } } peers { permanent_uuid: "2885bccd67924828bdf6f7034c144870" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37937 } } peers { permanent_uuid: "14791172465d4e268abedc032776781b" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38503 } }
I20260929 15:56:08.622951 2282 heartbeater.cc:503] Master 127.15.196.254:39265 was elected leader, sending a full tablet report...
I20260929 15:56:08.623636 2437 ts_tablet_manager.cc:1428] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260929 15:56:08.624614 2436 ts_tablet_manager.cc:1428] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260929 15:56:08.630786 2438 raft_consensus.cc:348] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5226fcd62c2498fb7882d1e9ceffae6" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39183 } } peers { permanent_uuid: "2885bccd67924828bdf6f7034c144870" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37937 } } peers { permanent_uuid: "14791172465d4e268abedc032776781b" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38503 } }
I20260929 15:56:08.631119 2438 raft_consensus.cc:374] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:08.631184 2438 raft_consensus.cc:729] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14791172465d4e268abedc032776781b, State: Initialized, Role: FOLLOWER
I20260929 15:56:08.631759 2438 consensus_queue.cc:260] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5226fcd62c2498fb7882d1e9ceffae6" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39183 } } peers { permanent_uuid: "2885bccd67924828bdf6f7034c144870" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37937 } } peers { permanent_uuid: "14791172465d4e268abedc032776781b" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38503 } }
I20260929 15:56:08.633261 2438 ts_tablet_manager.cc:1428] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260929 15:56:08.679418 2442 raft_consensus.cc:482] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:56:08.679728 2442 raft_consensus.cc:504] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5226fcd62c2498fb7882d1e9ceffae6" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39183 } } peers { permanent_uuid: "2885bccd67924828bdf6f7034c144870" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37937 } } peers { permanent_uuid: "14791172465d4e268abedc032776781b" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38503 } }
I20260929 15:56:08.681540 2442 leader_election.cc:288] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2885bccd67924828bdf6f7034c144870 (127.15.196.194:37937), 14791172465d4e268abedc032776781b (127.15.196.193:38503)
I20260929 15:56:08.688019 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:08.682144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:40323 (local address 127.15.196.194:37937)
0929 15:56:08.682335 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:08.682342 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:08.682476 (+ 134us) server_negotiation.cc:407] Connection header received
0929 15:56:08.682741 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:08.682747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:08.682837 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:08.682940 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:08.683481 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.684376 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.686090 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.687017 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.687127 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:08.687178 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:08.687572 (+ 394us) server_negotiation.cc:299] Negotiation successful
0929 15:56:08.687742 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 15:56:08.688036 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:08.681927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:37937 (local address 127.15.196.195:40323)
0929 15:56:08.682347 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:08.682378 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:08.682545 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:08.682989 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:08.682993 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:08.683106 (+ 113us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:08.683326 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:08.683344 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.684514 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.684520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:08.685926 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:08.685938 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.687166 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.687173 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:08.687393 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:08.687430 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 15:56:08.687541 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 15:56:08.687753 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"spinlock_wait_cycles":27008}
I20260929 15:56:08.688474 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:08.682060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:47389 (local address 127.15.196.193:38503)
0929 15:56:08.682275 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:08.682283 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:08.683008 (+ 725us) server_negotiation.cc:407] Connection header received
0929 15:56:08.683152 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:08.683157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:08.683231 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:08.683373 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:08.683960 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.684771 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.686562 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.687528 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.687678 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:08.687747 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:08.688073 (+ 326us) server_negotiation.cc:299] Negotiation successful
0929 15:56:08.688207 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 15:56:08.688506 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:08.681987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:38503 (local address 127.15.196.195:47389)
0929 15:56:08.682881 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:08.682900 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:08.683019 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:08.683386 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:08.683389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:08.683430 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:08.683804 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:08.683815 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.684928 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.684931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:08.686408 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:08.686422 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.687721 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.687727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:08.687898 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:08.687929 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 15:56:08.688039 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 15:56:08.688191 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"thread_start_us":493,"threads_started":1}
I20260929 15:56:08.689414 2237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d95661bcfe8b4320abde518b913d1efe" candidate_uuid: "a5226fcd62c2498fb7882d1e9ceffae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2885bccd67924828bdf6f7034c144870" is_pre_election: true
I20260929 15:56:08.689531 2102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d95661bcfe8b4320abde518b913d1efe" candidate_uuid: "a5226fcd62c2498fb7882d1e9ceffae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14791172465d4e268abedc032776781b" is_pre_election: true
I20260929 15:56:08.690022 2237 raft_consensus.cc:2454] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5226fcd62c2498fb7882d1e9ceffae6 in term 0.
I20260929 15:56:08.690022 2102 raft_consensus.cc:2454] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5226fcd62c2498fb7882d1e9ceffae6 in term 0.
I20260929 15:56:08.690765 2305 leader_election.cc:302] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 14791172465d4e268abedc032776781b, 2885bccd67924828bdf6f7034c144870, a5226fcd62c2498fb7882d1e9ceffae6; no voters:
I20260929 15:56:08.691181 2442 raft_consensus.cc:2781] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:56:08.691356 2442 raft_consensus.cc:482] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:56:08.691463 2442 raft_consensus.cc:3037] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:08.693661 2442 raft_consensus.cc:504] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5226fcd62c2498fb7882d1e9ceffae6" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39183 } } peers { permanent_uuid: "2885bccd67924828bdf6f7034c144870" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37937 } } peers { permanent_uuid: "14791172465d4e268abedc032776781b" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38503 } }
I20260929 15:56:08.694236 2442 leader_election.cc:288] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [CANDIDATE]: Term 1 election: Requested vote from peers 2885bccd67924828bdf6f7034c144870 (127.15.196.194:37937), 14791172465d4e268abedc032776781b (127.15.196.193:38503)
I20260929 15:56:08.694715 2237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d95661bcfe8b4320abde518b913d1efe" candidate_uuid: "a5226fcd62c2498fb7882d1e9ceffae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2885bccd67924828bdf6f7034c144870"
I20260929 15:56:08.694736 2102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d95661bcfe8b4320abde518b913d1efe" candidate_uuid: "a5226fcd62c2498fb7882d1e9ceffae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14791172465d4e268abedc032776781b"
I20260929 15:56:08.694931 2237 raft_consensus.cc:3037] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:08.694941 2102 raft_consensus.cc:3037] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b [term 0 FOLLOWER]: Advancing to term 1
W20260929 15:56:08.695566 2148 tablet.cc:2369] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:56:08.697276 2237 raft_consensus.cc:2454] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5226fcd62c2498fb7882d1e9ceffae6 in term 1.
I20260929 15:56:08.697638 2102 raft_consensus.cc:2454] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5226fcd62c2498fb7882d1e9ceffae6 in term 1.
I20260929 15:56:08.697724 2304 leader_election.cc:302] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2885bccd67924828bdf6f7034c144870, a5226fcd62c2498fb7882d1e9ceffae6; no voters:
I20260929 15:56:08.698057 2442 raft_consensus.cc:2781] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:08.698511 2442 raft_consensus.cc:686] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [term 1 LEADER]: Becoming Leader. State: Replica: a5226fcd62c2498fb7882d1e9ceffae6, State: Running, Role: LEADER
I20260929 15:56:08.698992 2442 consensus_queue.cc:237] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5226fcd62c2498fb7882d1e9ceffae6" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39183 } } peers { permanent_uuid: "2885bccd67924828bdf6f7034c144870" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37937 } } peers { permanent_uuid: "14791172465d4e268abedc032776781b" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38503 } }
I20260929 15:56:08.703953 1957 catalog_manager.cc:5495] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a5226fcd62c2498fb7882d1e9ceffae6 (127.15.196.195). New cstate: current_term: 1 leader_uuid: "a5226fcd62c2498fb7882d1e9ceffae6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5226fcd62c2498fb7882d1e9ceffae6" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2885bccd67924828bdf6f7034c144870" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14791172465d4e268abedc032776781b" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38503 } health_report { overall_health: UNKNOWN } } }
W20260929 15:56:08.718453 2283 tablet.cc:2369] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:56:08.731824 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:08.726237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52020 (local address 127.15.196.195:39183)
0929 15:56:08.726454 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:08.726460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:08.726577 (+ 117us) server_negotiation.cc:407] Connection header received
0929 15:56:08.726828 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:08.726832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:08.726919 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:08.727074 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:08.727545 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.728237 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.729623 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.730017 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.730146 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:08.730728 (+ 582us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:08.731231 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:08.731462 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:08.731546 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 15:56:08.731592 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 15:56:08.739866 2102 raft_consensus.cc:1261] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b [term 1 FOLLOWER]: Refusing update from remote peer a5226fcd62c2498fb7882d1e9ceffae6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:56:08.739903 2237 raft_consensus.cc:1261] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870 [term 1 FOLLOWER]: Refusing update from remote peer a5226fcd62c2498fb7882d1e9ceffae6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:56:08.740809 2446 consensus_queue.cc:1035] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2885bccd67924828bdf6f7034c144870" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 37937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:56:08.741202 2442 consensus_queue.cc:1035] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14791172465d4e268abedc032776781b" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 15:56:08.746057 2418 tablet.cc:2369] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:56:08.754853 2451 mvcc.cc:205] Tried to move back new op lower bound from 7334696422348787712 to 7334696422195470336. Current Snapshot: MvccSnapshot[applied={T|T < 7334696422348787712}]
I20260929 15:56:08.766539 1956 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:56:08.766888 1956 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:56:08.770535 1956 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c3b3fea79dc048e2bb518b0b3593ef92: Sending DeleteTablet for 3 replicas of tablet d95661bcfe8b4320abde518b913d1efe
I20260929 15:56:08.771699 2352 tablet_service.cc:1514] Processing DeleteTablet for tablet d95661bcfe8b4320abde518b913d1efe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:08 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:52016
I20260929 15:56:08.771807 2217 tablet_service.cc:1514] Processing DeleteTablet for tablet d95661bcfe8b4320abde518b913d1efe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:08 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:48266
I20260929 15:56:08.772166 2082 tablet_service.cc:1514] Processing DeleteTablet for tablet d95661bcfe8b4320abde518b913d1efe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:08 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:49074
I20260929 15:56:08.776170 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 2019
I20260929 15:56:08.783207 2460 tablet_replica.cc:330] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b: stopping tablet replica
I20260929 15:56:08.783849 2460 raft_consensus.cc:2229] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:56:08.784547 2460 raft_consensus.cc:2258] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:08.788039 2460 ts_tablet_manager.cc:1905] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:56:08.789951 2459 tablet_replica.cc:330] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870: stopping tablet replica
I20260929 15:56:08.790424 2458 tablet_replica.cc:330] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6: stopping tablet replica
I20260929 15:56:08.790482 2459 raft_consensus.cc:2229] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:56:08.790851 2458 raft_consensus.cc:2229] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:56:08.791015 2459 raft_consensus.cc:2258] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:08.791600 2458 raft_consensus.cc:2258] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:08.793692 2459 ts_tablet_manager.cc:1905] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 29 15:56:08 dist-test-slave-c51v krb5kdc[1912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:56:08.797993 2460 ts_tablet_manager.cc:1918] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:56:08.798202 2460 log.cc:1196] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/d95661bcfe8b4320abde518b913d1efe
I20260929 15:56:08.798898 2460 ts_tablet_manager.cc:1939] T d95661bcfe8b4320abde518b913d1efe P 14791172465d4e268abedc032776781b: Deleting consensus metadata
I20260929 15:56:08.801074 1942 catalog_manager.cc:4841] TS 14791172465d4e268abedc032776781b (127.15.196.193:38503): tablet d95661bcfe8b4320abde518b913d1efe (table test-table [id=9377f8cf56fd43e193c9fa0384e535cf]) successfully deleted
I20260929 15:56:08.802595 2459 ts_tablet_manager.cc:1918] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:56:08.802831 2459 log.cc:1196] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/d95661bcfe8b4320abde518b913d1efe
I20260929 15:56:08.803494 2458 ts_tablet_manager.cc:1905] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:56:08.803530 2459 ts_tablet_manager.cc:1939] T d95661bcfe8b4320abde518b913d1efe P 2885bccd67924828bdf6f7034c144870: Deleting consensus metadata
I20260929 15:56:08.805507 1941 catalog_manager.cc:4841] TS 2885bccd67924828bdf6f7034c144870 (127.15.196.194:37937): tablet d95661bcfe8b4320abde518b913d1efe (table test-table [id=9377f8cf56fd43e193c9fa0384e535cf]) successfully deleted
I20260929 15:56:08.805862 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:08.777142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49090 (local address 127.15.196.193:38503)
0929 15:56:08.777320 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:08.777326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:08.785530 (+ 8204us) server_negotiation.cc:407] Connection header received
0929 15:56:08.785776 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:08.785785 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:08.785926 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:08.786061 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:08.788532 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.789825 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.791551 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.792251 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.792343 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:08.798067 (+ 5724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:08.798087 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:08.798108 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:08.798161 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:08.802119 (+ 3958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:08.803040 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:08.803052 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:08.803057 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:08.803228 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:08.803768 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:08.803777 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:08.803781 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:08.804237 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:08.804516 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:08.805175 (+ 659us) server_negotiation.cc:299] Negotiation successful
0929 15:56:08.805403 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 15:56:08.807135 2142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:08.812328 2458 ts_tablet_manager.cc:1918] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:56:08.812515 2458 log.cc:1196] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/d95661bcfe8b4320abde518b913d1efe
I20260929 15:56:08.813166 2458 ts_tablet_manager.cc:1939] T d95661bcfe8b4320abde518b913d1efe P a5226fcd62c2498fb7882d1e9ceffae6: Deleting consensus metadata
I20260929 15:56:08.815025 1942 catalog_manager.cc:4841] TS a5226fcd62c2498fb7882d1e9ceffae6 (127.15.196.195:39183): tablet d95661bcfe8b4320abde518b913d1efe (table test-table [id=9377f8cf56fd43e193c9fa0384e535cf]) successfully deleted
I20260929 15:56:08.897850 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 2019
I20260929 15:56:08.921809 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 2153
Sep 29 15:56:08 dist-test-slave-c51v krb5kdc[1912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:56:08.938725 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:08.922676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.15.196.194:37937)
0929 15:56:08.922869 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:08.922873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:08.922891 (+ 18us) server_negotiation.cc:407] Connection header received
0929 15:56:08.923088 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:08.923092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:08.923163 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:08.923234 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:08.925137 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.925870 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.927901 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:08.928343 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:08.928480 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:08.932776 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:08.932791 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:08.932808 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:08.932844 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:08.935960 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:08.936663 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:08.936670 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:08.936672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:08.936791 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:08.937161 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:08.937170 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:08.937172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:08.937538 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:08.937837 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:08.938160 (+ 323us) server_negotiation.cc:299] Negotiation successful
0929 15:56:08.938400 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 15:56:08.939522 2277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:09.039624 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 2153
I20260929 15:56:09.066608 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 2288
Sep 29 15:56:09 dist-test-slave-c51v krb5kdc[1912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:56:09.084903 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:09.067842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52036 (local address 127.15.196.195:39183)
0929 15:56:09.068084 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:09.068091 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:09.068113 (+ 22us) server_negotiation.cc:407] Connection header received
0929 15:56:09.068201 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:09.068208 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:09.068300 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:09.068426 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:09.069640 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:09.071023 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:09.072366 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:09.073182 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:09.073325 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:09.078290 (+ 4965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:09.078308 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:09.078323 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:09.078365 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:09.081703 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:09.082662 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:09.082668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:09.082670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:09.082925 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:09.083400 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:09.083403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:09.083405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:09.083754 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:09.083978 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:09.084386 (+ 408us) server_negotiation.cc:299] Negotiation successful
0929 15:56:09.084552 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260929 15:56:09.085738 2412 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:09.206653 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 2288
I20260929 15:56:09.222368 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 1925
I20260929 15:56:09.223479 1987 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:09.335961 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 1925
2026-09-29T15:56:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4319 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-29T15:56:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:09Z Disabled control of system clock
I20260929 15:56:09.413300 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:34713
--webserver_interface=127.15.196.254
--webserver_port=0
--builtin_ntp_servers=127.15.196.212:32789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:34713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 15:56:09.754185 2469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:09.754487 2469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:09.754594 2469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:09.762126 2469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:09.762262 2469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:09.762319 2469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:09.762387 2469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:09.762437 2469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:09.772086 2469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:32789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:34713
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:34713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:09.773357 2469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:09.775162 2469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:09.787815 2474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:09.788298 2475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:09.788340 2477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:09.789515 2469 server_base.cc:1034] running on GCE node
I20260929 15:56:09.790616 2469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:09.792642 2469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:09.793900 2469 hybrid_clock.cc:648] HybridClock initialized: now 1790697369793810 us; error 94 us; skew 500 ppm
I20260929 15:56:09.794247 2469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:56:09.800673 2469 webserver.cc:460] Webserver started at http://127.15.196.254:35523/ using document root <none> and password file <none>
I20260929 15:56:09.801373 2469 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:09.801467 2469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:09.801752 2469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:09.804769 2469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "de37c09f51624dbaa38147c7fc5ec458"
format_stamp: "Formatted at 2026-09-29 15:56:09 on dist-test-slave-c51v"
server_key: "2867ecfeb18bd568bfb0aa48a331428b"
server_key_iv: "094736d334d80a42c278462d4a846fc7"
server_key_version: "encryptionkey@0"
I20260929 15:56:09.805567 2469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "de37c09f51624dbaa38147c7fc5ec458"
format_stamp: "Formatted at 2026-09-29 15:56:09 on dist-test-slave-c51v"
server_key: "2867ecfeb18bd568bfb0aa48a331428b"
server_key_iv: "094736d334d80a42c278462d4a846fc7"
server_key_version: "encryptionkey@0"
I20260929 15:56:09.811012 2469 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 15:56:09.814337 2483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:09.815764 2469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:09.815912 2469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "de37c09f51624dbaa38147c7fc5ec458"
format_stamp: "Formatted at 2026-09-29 15:56:09 on dist-test-slave-c51v"
server_key: "2867ecfeb18bd568bfb0aa48a331428b"
server_key_iv: "094736d334d80a42c278462d4a846fc7"
server_key_version: "encryptionkey@0"
I20260929 15:56:09.816082 2469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:09.833797 2469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:09.834525 2469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:09.834744 2469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:09.852767 2469 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:34713
I20260929 15:56:09.852779 2534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:34713 every 8 connection(s)
I20260929 15:56:09.854127 2469 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:09.857365 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 2469
I20260929 15:56:09.857551 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:09.858006 16147 external_mini_cluster.cc:1428] Setting key 024dc6d49ba1ff42959a8062891b68a1
I20260929 15:56:09.858057 2535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:09.873240 2538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:09.860001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44028 (local address 127.15.196.254:34713)
0929 15:56:09.861079 (+ 1078us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:09.861100 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:09.861154 (+ 54us) server_negotiation.cc:407] Connection header received
0929 15:56:09.863077 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:09.863108 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:09.863748 (+ 640us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:09.864186 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:09.864733 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:09.866866 (+ 2133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:09.868247 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:09.869282 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:09.869481 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:09.870508 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:09.870519 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:09.870539 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:09.871252 (+ 713us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:09.871382 (+ 130us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:09.871412 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:09.871525 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:09.871709 (+ 184us) server_negotiation.cc:299] Negotiation successful
0929 15:56:09.872014 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":349,"threads_started":1}
I20260929 15:56:09.873648 2535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458: Bootstrap starting.
I20260929 15:56:09.876894 2535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:09.877898 2535 log.cc:824] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:09.880578 2535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458: No bootstrap required, opened a new log
I20260929 15:56:09.888024 2535 raft_consensus.cc:348] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de37c09f51624dbaa38147c7fc5ec458" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34713 } }
I20260929 15:56:09.888768 2535 raft_consensus.cc:374] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:09.888906 2535 raft_consensus.cc:729] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de37c09f51624dbaa38147c7fc5ec458, State: Initialized, Role: FOLLOWER
I20260929 15:56:09.889590 2535 consensus_queue.cc:260] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de37c09f51624dbaa38147c7fc5ec458" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34713 } }
I20260929 15:56:09.889868 2535 raft_consensus.cc:388] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:09.890033 2535 raft_consensus.cc:482] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:09.890224 2535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:09.892655 2535 raft_consensus.cc:504] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de37c09f51624dbaa38147c7fc5ec458" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34713 } }
I20260929 15:56:09.893160 2535 leader_election.cc:302] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de37c09f51624dbaa38147c7fc5ec458; no voters:
I20260929 15:56:09.893852 2535 leader_election.cc:288] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:09.894039 2540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:09.894752 2540 raft_consensus.cc:686] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [term 1 LEADER]: Becoming Leader. State: Replica: de37c09f51624dbaa38147c7fc5ec458, State: Running, Role: LEADER
I20260929 15:56:09.895279 2540 consensus_queue.cc:237] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de37c09f51624dbaa38147c7fc5ec458" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34713 } }
I20260929 15:56:09.896220 2535 sys_catalog.cc:564] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:09.898293 2542 sys_catalog.cc:455] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de37c09f51624dbaa38147c7fc5ec458. Latest consensus state: current_term: 1 leader_uuid: "de37c09f51624dbaa38147c7fc5ec458" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de37c09f51624dbaa38147c7fc5ec458" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34713 } } }
I20260929 15:56:09.898693 2542 sys_catalog.cc:458] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:09.899117 2541 sys_catalog.cc:455] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de37c09f51624dbaa38147c7fc5ec458" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de37c09f51624dbaa38147c7fc5ec458" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34713 } } }
I20260929 15:56:09.899541 2541 sys_catalog.cc:458] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:09.904326 2545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:09.911285 2545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:09.920838 2545 catalog_manager.cc:1347] Generated new cluster ID: 784ec71b58664da8a847f206fe419935
I20260929 15:56:09.920966 2545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:09.936684 2545 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:09.938467 2545 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:09.949043 2545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458: Generated new TSK 0
I20260929 15:56:09.950103 2545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:09.967969 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34713
--builtin_ntp_servers=127.15.196.212:32789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 15:56:10.301292 2559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:10.301578 2559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:10.301666 2559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:10.308784 2559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:10.308889 2559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:10.308995 2559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:56:10.318687 2559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:32789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:10.320149 2559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:10.322002 2559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:10.335016 2564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:10.335090 2567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:10.334978 2565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:10.336018 2559 server_base.cc:1034] running on GCE node
I20260929 15:56:10.336673 2559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:10.337591 2559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:10.338840 2559 hybrid_clock.cc:648] HybridClock initialized: now 1790697370338792 us; error 60 us; skew 500 ppm
I20260929 15:56:10.339174 2559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:56:10.341540 2559 webserver.cc:460] Webserver started at http://127.15.196.193:36055/ using document root <none> and password file <none>
I20260929 15:56:10.342244 2559 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:10.342372 2559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:10.342687 2559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:10.345603 2559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "edf38a1caa55469b85c8eadd06550544"
format_stamp: "Formatted at 2026-09-29 15:56:10 on dist-test-slave-c51v"
server_key: "da1e47d00738312834b52c9074f962c0"
server_key_iv: "0f57ebbbf828564d1a7be1d7526bf923"
server_key_version: "encryptionkey@0"
I20260929 15:56:10.346429 2559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "edf38a1caa55469b85c8eadd06550544"
format_stamp: "Formatted at 2026-09-29 15:56:10 on dist-test-slave-c51v"
server_key: "da1e47d00738312834b52c9074f962c0"
server_key_iv: "0f57ebbbf828564d1a7be1d7526bf923"
server_key_version: "encryptionkey@0"
I20260929 15:56:10.351660 2559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 15:56:10.355062 2573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:10.356467 2559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:10.356647 2559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "edf38a1caa55469b85c8eadd06550544"
format_stamp: "Formatted at 2026-09-29 15:56:10 on dist-test-slave-c51v"
server_key: "da1e47d00738312834b52c9074f962c0"
server_key_iv: "0f57ebbbf828564d1a7be1d7526bf923"
server_key_version: "encryptionkey@0"
I20260929 15:56:10.356877 2559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:10.370198 2559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:10.371028 2559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:10.371417 2559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:10.372462 2559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:10.373989 2559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:10.374104 2559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:10.374233 2559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:10.374331 2559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:10.408221 2559 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:41019
I20260929 15:56:10.408325 2685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:41019 every 8 connection(s)
I20260929 15:56:10.409690 2559 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:56:10.412209 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 2559
I20260929 15:56:10.412370 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:56:10.412753 16147 external_mini_cluster.cc:1428] Setting key f0346dfa2d121b021e9f06ba5ed348ea
I20260929 15:56:10.415757 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34713
--builtin_ntp_servers=127.15.196.212:32789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 15:56:10.423997 2689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:10.413390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:56269 (local address 127.15.196.254:34713)
0929 15:56:10.413832 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:10.413838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:10.414195 (+ 357us) server_negotiation.cc:407] Connection header received
0929 15:56:10.416097 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:10.416102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:10.416163 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:10.416243 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:10.417849 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:10.418579 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:10.421116 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:10.421543 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:10.421640 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:10.423225 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:10.423230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:10.423233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:10.423251 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:10.423335 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:10.423358 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:10.423425 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:10.423665 (+ 240us) server_negotiation.cc:299] Negotiation successful
0929 15:56:10.423743 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":216,"threads_started":1}
I20260929 15:56:10.425256 2688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:10.412944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:34713 (local address 127.15.196.193:56269)
0929 15:56:10.414026 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:10.414073 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:10.415733 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:10.416551 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:10.416564 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:10.416702 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:10.417586 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:10.417612 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:10.418749 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:10.418756 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:10.420974 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:10.420993 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:10.421777 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:10.421786 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:10.422115 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:10.422872 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:56:10.422878 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:10.422939 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:56:10.423039 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:10.423448 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:10.423461 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:10.423471 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 15:56:10.423661 (+ 190us) client_negotiation.cc:240] Negotiation successful
0929 15:56:10.424083 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":349,"threads_started":1}
I20260929 15:56:10.426999 2686 heartbeater.cc:347] Connected to a master server at 127.15.196.254:34713
I20260929 15:56:10.427371 2686 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:10.428110 2686 heartbeater.cc:511] Master 127.15.196.254:34713 requested a full tablet report, sending...
I20260929 15:56:10.430099 2500 ts_manager.cc:194] Registered new tserver with Master: edf38a1caa55469b85c8eadd06550544 (127.15.196.193:41019)
I20260929 15:56:10.431670 2500 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.196.193:56269
W20260929 15:56:10.744834 2690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:10.745146 2690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:10.745311 2690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:10.752388 2690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:10.752522 2690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:10.752738 2690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:56:10.762013 2690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:32789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:10.763494 2690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:10.765311 2690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:10.778541 2698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:10.778576 2696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:10.778605 2695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:10.779345 2690 server_base.cc:1034] running on GCE node
I20260929 15:56:10.780011 2690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:10.781028 2690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:10.782258 2690 hybrid_clock.cc:648] HybridClock initialized: now 1790697370782210 us; error 44 us; skew 500 ppm
I20260929 15:56:10.782581 2690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:56:10.784919 2690 webserver.cc:460] Webserver started at http://127.15.196.194:44091/ using document root <none> and password file <none>
I20260929 15:56:10.785634 2690 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:10.785777 2690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:10.786092 2690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:10.789021 2690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "42be2478c07e4188bb6dce9855f8037f"
format_stamp: "Formatted at 2026-09-29 15:56:10 on dist-test-slave-c51v"
server_key: "dc80d88df95d77e2093618eaf02a0389"
server_key_iv: "71937d2ce4531b93295a8bc87cee6820"
server_key_version: "encryptionkey@0"
I20260929 15:56:10.789825 2690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "42be2478c07e4188bb6dce9855f8037f"
format_stamp: "Formatted at 2026-09-29 15:56:10 on dist-test-slave-c51v"
server_key: "dc80d88df95d77e2093618eaf02a0389"
server_key_iv: "71937d2ce4531b93295a8bc87cee6820"
server_key_version: "encryptionkey@0"
I20260929 15:56:10.794904 2690 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260929 15:56:10.798360 2704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:10.799824 2690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:10.800001 2690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "42be2478c07e4188bb6dce9855f8037f"
format_stamp: "Formatted at 2026-09-29 15:56:10 on dist-test-slave-c51v"
server_key: "dc80d88df95d77e2093618eaf02a0389"
server_key_iv: "71937d2ce4531b93295a8bc87cee6820"
server_key_version: "encryptionkey@0"
I20260929 15:56:10.800220 2690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:10.824050 2690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:10.824937 2690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:10.825253 2690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:10.826242 2690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:10.827886 2690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:10.828004 2690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:10.828135 2690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:10.828235 2690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:10.860823 2690 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:41767
I20260929 15:56:10.860880 2818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:41767 every 8 connection(s)
I20260929 15:56:10.862358 2690 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:56:10.863442 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 2690
I20260929 15:56:10.863626 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:56:10.864183 16147 external_mini_cluster.cc:1428] Setting key f6aaf2a7d3775dc8231c32c0da0029a3
I20260929 15:56:10.867548 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34713
--builtin_ntp_servers=127.15.196.212:32789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 15:56:10.875797 2689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:10.866035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:44933 (local address 127.15.196.254:34713)
0929 15:56:10.866249 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:10.866257 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:10.867371 (+ 1114us) server_negotiation.cc:407] Connection header received
0929 15:56:10.869303 (+ 1932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:10.869309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:10.869393 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:10.869512 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:10.870609 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:10.871430 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:10.872971 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:10.873409 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:10.873550 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:10.874940 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:10.874947 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:10.874951 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:10.874977 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:10.875008 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:10.875021 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:10.875117 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:10.875383 (+ 266us) server_negotiation.cc:299] Negotiation successful
0929 15:56:10.875480 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 15:56:10.877043 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:10.866051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:34713 (local address 127.15.196.194:44933)
0929 15:56:10.867205 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:10.867251 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:10.869015 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:10.869758 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:10.869766 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:10.869865 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:10.870434 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:10.870451 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:10.871578 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:10.871584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:10.872820 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:10.872831 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:10.873636 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:10.873644 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:10.873879 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:10.874612 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:56:10.874615 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:10.874653 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:56:10.874744 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:10.875116 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:10.875128 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:10.875138 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 15:56:10.875338 (+ 200us) client_negotiation.cc:240] Negotiation successful
0929 15:56:10.875749 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":376,"threads_started":1}
I20260929 15:56:10.878659 2819 heartbeater.cc:347] Connected to a master server at 127.15.196.254:34713
I20260929 15:56:10.878970 2819 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:10.879876 2819 heartbeater.cc:511] Master 127.15.196.254:34713 requested a full tablet report, sending...
I20260929 15:56:10.881593 2500 ts_manager.cc:194] Registered new tserver with Master: 42be2478c07e4188bb6dce9855f8037f (127.15.196.194:41767)
I20260929 15:56:10.882810 2500 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.196.194:44933
W20260929 15:56:11.199718 2822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:11.200006 2822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:11.200098 2822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:11.207722 2822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:11.207834 2822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:11.207954 2822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:56:11.217095 2822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:32789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:34713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:11.218317 2822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:11.220293 2822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:11.233798 2830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:11.234203 2827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:11.233860 2828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:11.234719 2822 server_base.cc:1034] running on GCE node
I20260929 15:56:11.235368 2822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:11.236363 2822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:11.237577 2822 hybrid_clock.cc:648] HybridClock initialized: now 1790697371237526 us; error 43 us; skew 500 ppm
I20260929 15:56:11.237944 2822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:56:11.240338 2822 webserver.cc:460] Webserver started at http://127.15.196.195:44905/ using document root <none> and password file <none>
I20260929 15:56:11.241052 2822 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:11.241165 2822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:11.241492 2822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:11.244499 2822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "652adab41e964d87a85c0200917eb8d4"
format_stamp: "Formatted at 2026-09-29 15:56:11 on dist-test-slave-c51v"
server_key: "e4310fad1446298d85260af882c7ad0c"
server_key_iv: "37db68228fb79c7949878a3da4b82313"
server_key_version: "encryptionkey@0"
I20260929 15:56:11.245318 2822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "652adab41e964d87a85c0200917eb8d4"
format_stamp: "Formatted at 2026-09-29 15:56:11 on dist-test-slave-c51v"
server_key: "e4310fad1446298d85260af882c7ad0c"
server_key_iv: "37db68228fb79c7949878a3da4b82313"
server_key_version: "encryptionkey@0"
I20260929 15:56:11.250406 2822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 15:56:11.253821 2836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:11.255110 2822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:56:11.255267 2822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "652adab41e964d87a85c0200917eb8d4"
format_stamp: "Formatted at 2026-09-29 15:56:11 on dist-test-slave-c51v"
server_key: "e4310fad1446298d85260af882c7ad0c"
server_key_iv: "37db68228fb79c7949878a3da4b82313"
server_key_version: "encryptionkey@0"
I20260929 15:56:11.255527 2822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:11.289691 2822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:11.290584 2822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:11.290899 2822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:11.291916 2822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:11.293395 2822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:11.293509 2822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:11.293660 2822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:11.293751 2822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:11.326845 2822 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:46427
I20260929 15:56:11.326929 2950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:46427 every 8 connection(s)
I20260929 15:56:11.328226 2822 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:56:11.335886 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 2822
I20260929 15:56:11.336086 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:56:11.336642 16147 external_mini_cluster.cc:1428] Setting key ce1b25873e6c03a7af0c20d2a8ed8726
I20260929 15:56:11.341120 2689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.330653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:59931 (local address 127.15.196.254:34713)
0929 15:56:11.330948 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:11.330954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:11.332339 (+ 1385us) server_negotiation.cc:407] Connection header received
0929 15:56:11.334047 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:11.334052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:11.334125 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:11.334236 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:11.335231 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.336194 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.338354 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.338993 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.339120 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:11.340475 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:11.340480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:11.340483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:11.340499 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:11.340531 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:11.340543 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:11.340612 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:11.340814 (+ 202us) server_negotiation.cc:299] Negotiation successful
0929 15:56:11.340888 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260929 15:56:11.342046 2953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.331262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:34713 (local address 127.15.196.195:59931)
0929 15:56:11.332188 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:11.332229 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:11.333763 (+ 1534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:11.334406 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:11.334413 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:11.334493 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:11.335067 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:11.335082 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.336379 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.336385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:11.338197 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:11.338208 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.339207 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.339215 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:11.339548 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:11.340188 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:56:11.340190 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:11.340226 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:56:11.340299 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:11.340626 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:11.340633 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:11.340639 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 15:56:11.340769 (+ 130us) client_negotiation.cc:240] Negotiation successful
0929 15:56:11.341045 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":252,"threads_started":1}
I20260929 15:56:11.343364 2951 heartbeater.cc:347] Connected to a master server at 127.15.196.254:34713
I20260929 15:56:11.343642 2951 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:11.344337 2951 heartbeater.cc:511] Master 127.15.196.254:34713 requested a full tablet report, sending...
I20260929 15:56:11.345628 2500 ts_manager.cc:194] Registered new tserver with Master: 652adab41e964d87a85c0200917eb8d4 (127.15.196.195:46427)
I20260929 15:56:11.346351 2500 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.196.195:59931
I20260929 15:56:11.351613 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:56:11.361516 2689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.355640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44040 (local address 127.15.196.254:34713)
0929 15:56:11.355827 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:11.355833 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:11.356358 (+ 525us) server_negotiation.cc:407] Connection header received
0929 15:56:11.356505 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:11.356508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:11.356576 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:11.356647 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:11.357066 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.357753 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.359079 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.359565 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.359704 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:11.360849 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:11.360854 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:11.360858 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:11.360875 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:11.360907 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:11.360919 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:11.361025 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:11.361178 (+ 153us) server_negotiation.cc:299] Negotiation successful
0929 15:56:11.361221 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 15:56:11.365136 2500 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44040:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:56:11.372018 2500 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:56:11.402128 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.384123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49308 (local address 127.15.196.193:41019)
0929 15:56:11.385494 (+ 1371us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:11.385505 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:11.391446 (+ 5941us) server_negotiation.cc:407] Connection header received
0929 15:56:11.392425 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:11.392439 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:11.392842 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:11.393066 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:11.394301 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.396097 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.399104 (+ 3007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.400550 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.400698 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:11.400965 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:11.401603 (+ 638us) server_negotiation.cc:299] Negotiation successful
0929 15:56:11.401791 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1202,"thread_start_us":1083,"threads_started":1}
I20260929 15:56:11.402734 2963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.383960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:41019 (local address 127.0.0.1:49308)
0929 15:56:11.391275 (+ 7315us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:11.391339 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:11.392136 (+ 797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:11.393461 (+ 1325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:11.393471 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:11.393514 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:11.394066 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:11.394083 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.396338 (+ 2255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.396342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:11.398902 (+ 2560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:11.398918 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.401095 (+ 2177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.401103 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:11.401331 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:11.401382 (+ 51us) client_negotiation.cc:769] Sending connection context
0929 15:56:11.402089 (+ 707us) client_negotiation.cc:240] Negotiation successful
0929 15:56:11.402398 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6979,"thread_start_us":525,"threads_started":1}
I20260929 15:56:11.405176 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.384783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:46427 (local address 127.0.0.1:51844)
0929 15:56:11.392206 (+ 7423us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:11.392230 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:11.392382 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:11.397981 (+ 5599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:11.397985 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:11.398010 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:11.398500 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:11.398517 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.400068 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.400075 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:11.401892 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:11.401904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.404353 (+ 2449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.404360 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:11.404542 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:11.404584 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 15:56:11.404684 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 15:56:11.404814 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7240,"spinlock_wait_cycles":20352,"thread_start_us":7054,"threads_started":1}
I20260929 15:56:11.405624 2967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.385721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51844 (local address 127.15.196.195:46427)
0929 15:56:11.397439 (+ 11718us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:11.397447 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:11.397483 (+ 36us) server_negotiation.cc:407] Connection header received
0929 15:56:11.397574 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:11.397582 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:11.397796 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:11.397996 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:11.398726 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.399936 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.402071 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.403838 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.404033 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:11.404230 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:11.404953 (+ 723us) server_negotiation.cc:299] Negotiation successful
0929 15:56:11.405123 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11406,"thread_start_us":11223,"threads_started":1}
I20260929 15:56:11.405948 2968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.385908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49200 (local address 127.15.196.194:41767)
0929 15:56:11.397693 (+ 11785us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:11.397701 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:11.397730 (+ 29us) server_negotiation.cc:407] Connection header received
0929 15:56:11.397820 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:11.397827 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:11.398042 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:11.398214 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:11.398950 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.400500 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.402987 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.404476 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.404655 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:11.404849 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:11.405443 (+ 594us) server_negotiation.cc:299] Negotiation successful
0929 15:56:11.405610 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11611,"thread_start_us":11496,"threads_started":1}
I20260929 15:56:11.406384 2966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.385668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:41767 (local address 127.0.0.1:49200)
0929 15:56:11.396478 (+ 10810us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:11.396501 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:11.396653 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:11.398294 (+ 1641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:11.398300 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:11.398331 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:11.398788 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:11.398800 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.400683 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.400688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:11.402752 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:11.402765 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.404679 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.404685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:11.404877 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:11.404907 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 15:56:11.404999 (+ 92us) client_negotiation.cc:240] Negotiation successful
0929 15:56:11.405131 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10656,"mutex_wait_us":3,"thread_start_us":10429,"threads_started":1}
I20260929 15:56:11.407163 2621 tablet_service.cc:1467] Processing CreateTablet for tablet 09b5e7c6338a4f849c69595a178cfe98 (DEFAULT_TABLE table=test-table [id=c89de71f4a184cd08080d9fd422390ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:11.408612 2621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09b5e7c6338a4f849c69595a178cfe98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:11.409041 2885 tablet_service.cc:1467] Processing CreateTablet for tablet 09b5e7c6338a4f849c69595a178cfe98 (DEFAULT_TABLE table=test-table [id=c89de71f4a184cd08080d9fd422390ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:11.410161 2885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09b5e7c6338a4f849c69595a178cfe98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:11.417321 2753 tablet_service.cc:1467] Processing CreateTablet for tablet 09b5e7c6338a4f849c69595a178cfe98 (DEFAULT_TABLE table=test-table [id=c89de71f4a184cd08080d9fd422390ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:11.418613 2753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09b5e7c6338a4f849c69595a178cfe98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:11.426296 2969 tablet_bootstrap.cc:492] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4: Bootstrap starting.
I20260929 15:56:11.430266 2970 tablet_bootstrap.cc:492] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544: Bootstrap starting.
I20260929 15:56:11.430604 2969 tablet_bootstrap.cc:654] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:11.431892 2969 log.cc:824] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:11.432454 2971 tablet_bootstrap.cc:492] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f: Bootstrap starting.
I20260929 15:56:11.433807 2970 tablet_bootstrap.cc:654] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:11.434468 2686 heartbeater.cc:503] Master 127.15.196.254:34713 was elected leader, sending a full tablet report...
I20260929 15:56:11.435073 2970 log.cc:824] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:11.436273 2971 tablet_bootstrap.cc:654] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:11.436447 2969 tablet_bootstrap.cc:492] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4: No bootstrap required, opened a new log
I20260929 15:56:11.436719 2969 ts_tablet_manager.cc:1397] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260929 15:56:11.437808 2971 log.cc:824] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:11.444072 2971 tablet_bootstrap.cc:492] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f: No bootstrap required, opened a new log
I20260929 15:56:11.444353 2970 tablet_bootstrap.cc:492] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544: No bootstrap required, opened a new log
I20260929 15:56:11.444361 2971 ts_tablet_manager.cc:1397] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260929 15:56:11.444619 2970 ts_tablet_manager.cc:1397] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.005s
I20260929 15:56:11.449667 2969 raft_consensus.cc:348] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf38a1caa55469b85c8eadd06550544" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 41019 } } peers { permanent_uuid: "42be2478c07e4188bb6dce9855f8037f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 41767 } } peers { permanent_uuid: "652adab41e964d87a85c0200917eb8d4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46427 } }
I20260929 15:56:11.450189 2969 raft_consensus.cc:374] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:11.450294 2969 raft_consensus.cc:729] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652adab41e964d87a85c0200917eb8d4, State: Initialized, Role: FOLLOWER
I20260929 15:56:11.451007 2969 consensus_queue.cc:260] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf38a1caa55469b85c8eadd06550544" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 41019 } } peers { permanent_uuid: "42be2478c07e4188bb6dce9855f8037f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 41767 } } peers { permanent_uuid: "652adab41e964d87a85c0200917eb8d4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46427 } }
I20260929 15:56:11.452188 2951 heartbeater.cc:503] Master 127.15.196.254:34713 was elected leader, sending a full tablet report...
I20260929 15:56:11.452602 2971 raft_consensus.cc:348] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf38a1caa55469b85c8eadd06550544" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 41019 } } peers { permanent_uuid: "42be2478c07e4188bb6dce9855f8037f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 41767 } } peers { permanent_uuid: "652adab41e964d87a85c0200917eb8d4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46427 } }
I20260929 15:56:11.452981 2969 ts_tablet_manager.cc:1428] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260929 15:56:11.453058 2971 raft_consensus.cc:374] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:11.453143 2971 raft_consensus.cc:729] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42be2478c07e4188bb6dce9855f8037f, State: Initialized, Role: FOLLOWER
I20260929 15:56:11.453778 2971 consensus_queue.cc:260] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf38a1caa55469b85c8eadd06550544" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 41019 } } peers { permanent_uuid: "42be2478c07e4188bb6dce9855f8037f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 41767 } } peers { permanent_uuid: "652adab41e964d87a85c0200917eb8d4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46427 } }
I20260929 15:56:11.455930 2819 heartbeater.cc:503] Master 127.15.196.254:34713 was elected leader, sending a full tablet report...
I20260929 15:56:11.455832 2970 raft_consensus.cc:348] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf38a1caa55469b85c8eadd06550544" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 41019 } } peers { permanent_uuid: "42be2478c07e4188bb6dce9855f8037f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 41767 } } peers { permanent_uuid: "652adab41e964d87a85c0200917eb8d4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46427 } }
I20260929 15:56:11.456262 2970 raft_consensus.cc:374] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:11.456364 2970 raft_consensus.cc:729] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf38a1caa55469b85c8eadd06550544, State: Initialized, Role: FOLLOWER
I20260929 15:56:11.456564 2971 ts_tablet_manager.cc:1428] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260929 15:56:11.456977 2970 consensus_queue.cc:260] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf38a1caa55469b85c8eadd06550544" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 41019 } } peers { permanent_uuid: "42be2478c07e4188bb6dce9855f8037f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 41767 } } peers { permanent_uuid: "652adab41e964d87a85c0200917eb8d4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46427 } }
I20260929 15:56:11.458850 2970 ts_tablet_manager.cc:1428] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
W20260929 15:56:11.579874 2952 tablet.cc:2369] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:56:11.608842 2975 raft_consensus.cc:482] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:56:11.609272 2975 raft_consensus.cc:504] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf38a1caa55469b85c8eadd06550544" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 41019 } } peers { permanent_uuid: "42be2478c07e4188bb6dce9855f8037f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 41767 } } peers { permanent_uuid: "652adab41e964d87a85c0200917eb8d4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46427 } }
I20260929 15:56:11.611070 2975 leader_election.cc:288] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edf38a1caa55469b85c8eadd06550544 (127.15.196.193:41019), 42be2478c07e4188bb6dce9855f8037f (127.15.196.194:41767)
W20260929 15:56:11.616183 2820 tablet.cc:2369] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:56:11.617846 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.611570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:59155 (local address 127.15.196.193:41019)
0929 15:56:11.611783 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:11.611791 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:11.611815 (+ 24us) server_negotiation.cc:407] Connection header received
0929 15:56:11.612368 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:11.612375 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:11.612473 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:11.612581 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:11.613057 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.614020 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.615974 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.616955 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.617117 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:11.617195 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:11.617458 (+ 263us) server_negotiation.cc:299] Negotiation successful
0929 15:56:11.617580 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 15:56:11.617856 2953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.611494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:41019 (local address 127.15.196.195:59155)
0929 15:56:11.611720 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:11.611742 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:11.611888 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:11.612607 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:11.612610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:11.612663 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:11.612897 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:11.612909 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.614146 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.614152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:11.615811 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:11.615823 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.617110 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.617116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:11.617285 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:11.617310 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 15:56:11.617429 (+ 119us) client_negotiation.cc:240] Negotiation successful
0929 15:56:11.617565 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":1}
I20260929 15:56:11.618484 2968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.612189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:46319 (local address 127.15.196.194:41767)
0929 15:56:11.612415 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:11.612420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:11.612781 (+ 361us) server_negotiation.cc:407] Connection header received
0929 15:56:11.612932 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:11.612938 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:11.613033 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:11.613116 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:11.613700 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.614552 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.616523 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.617521 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.617687 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:11.617753 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:11.618075 (+ 322us) server_negotiation.cc:299] Negotiation successful
0929 15:56:11.618207 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 15:56:11.618953 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.611938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:41767 (local address 127.15.196.195:46319)
0929 15:56:11.612674 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:11.612693 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:11.612809 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:11.613165 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:11.613169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:11.613193 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:11.613543 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:11.613554 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.614695 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.614699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:11.616258 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:11.616272 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.617687 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.617693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:11.617869 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:11.617890 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 15:56:11.618629 (+ 739us) client_negotiation.cc:240] Negotiation successful
0929 15:56:11.618736 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":364,"threads_started":1}
I20260929 15:56:11.619144 2641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09b5e7c6338a4f849c69595a178cfe98" candidate_uuid: "652adab41e964d87a85c0200917eb8d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf38a1caa55469b85c8eadd06550544" is_pre_election: true
I20260929 15:56:11.619762 2641 raft_consensus.cc:2454] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652adab41e964d87a85c0200917eb8d4 in term 0.
I20260929 15:56:11.620057 2773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09b5e7c6338a4f849c69595a178cfe98" candidate_uuid: "652adab41e964d87a85c0200917eb8d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42be2478c07e4188bb6dce9855f8037f" is_pre_election: true
I20260929 15:56:11.620522 2839 leader_election.cc:302] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652adab41e964d87a85c0200917eb8d4, edf38a1caa55469b85c8eadd06550544; no voters:
I20260929 15:56:11.620832 2773 raft_consensus.cc:2454] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652adab41e964d87a85c0200917eb8d4 in term 0.
I20260929 15:56:11.620994 2975 raft_consensus.cc:2781] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:56:11.621142 2975 raft_consensus.cc:482] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:56:11.621246 2975 raft_consensus.cc:3037] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:11.623600 2975 raft_consensus.cc:504] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf38a1caa55469b85c8eadd06550544" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 41019 } } peers { permanent_uuid: "42be2478c07e4188bb6dce9855f8037f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 41767 } } peers { permanent_uuid: "652adab41e964d87a85c0200917eb8d4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46427 } }
I20260929 15:56:11.624101 2975 leader_election.cc:288] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [CANDIDATE]: Term 1 election: Requested vote from peers edf38a1caa55469b85c8eadd06550544 (127.15.196.193:41019), 42be2478c07e4188bb6dce9855f8037f (127.15.196.194:41767)
I20260929 15:56:11.624584 2641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09b5e7c6338a4f849c69595a178cfe98" candidate_uuid: "652adab41e964d87a85c0200917eb8d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf38a1caa55469b85c8eadd06550544"
I20260929 15:56:11.624666 2773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09b5e7c6338a4f849c69595a178cfe98" candidate_uuid: "652adab41e964d87a85c0200917eb8d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42be2478c07e4188bb6dce9855f8037f"
I20260929 15:56:11.624791 2641 raft_consensus.cc:3037] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:11.624899 2773 raft_consensus.cc:3037] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:11.627034 2773 raft_consensus.cc:2454] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652adab41e964d87a85c0200917eb8d4 in term 1.
I20260929 15:56:11.627038 2641 raft_consensus.cc:2454] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652adab41e964d87a85c0200917eb8d4 in term 1.
I20260929 15:56:11.627421 2839 leader_election.cc:302] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42be2478c07e4188bb6dce9855f8037f, 652adab41e964d87a85c0200917eb8d4; no voters:
I20260929 15:56:11.627715 2975 raft_consensus.cc:2781] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:11.628170 2975 raft_consensus.cc:686] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [term 1 LEADER]: Becoming Leader. State: Replica: 652adab41e964d87a85c0200917eb8d4, State: Running, Role: LEADER
I20260929 15:56:11.628643 2975 consensus_queue.cc:237] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf38a1caa55469b85c8eadd06550544" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 41019 } } peers { permanent_uuid: "42be2478c07e4188bb6dce9855f8037f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 41767 } } peers { permanent_uuid: "652adab41e964d87a85c0200917eb8d4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46427 } }
I20260929 15:56:11.633005 2499 catalog_manager.cc:5495] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652adab41e964d87a85c0200917eb8d4 (127.15.196.195). New cstate: current_term: 1 leader_uuid: "652adab41e964d87a85c0200917eb8d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf38a1caa55469b85c8eadd06550544" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 41019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42be2478c07e4188bb6dce9855f8037f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 41767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652adab41e964d87a85c0200917eb8d4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 46427 } health_report { overall_health: HEALTHY } } }
W20260929 15:56:11.662189 2687 tablet.cc:2369] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:56:11.691699 2967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.686185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51852 (local address 127.15.196.195:46427)
0929 15:56:11.686415 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:11.686420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:11.686439 (+ 19us) server_negotiation.cc:407] Connection header received
0929 15:56:11.686624 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:11.686629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:11.686720 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:11.686873 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:11.687307 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.688062 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.689446 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.689871 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.690005 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:11.690596 (+ 591us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:11.691098 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:11.691290 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:11.691422 (+ 132us) server_negotiation.cc:299] Negotiation successful
0929 15:56:11.691466 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 15:56:11.700011 2641 raft_consensus.cc:1261] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544 [term 1 FOLLOWER]: Refusing update from remote peer 652adab41e964d87a85c0200917eb8d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:56:11.700052 2773 raft_consensus.cc:1261] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f [term 1 FOLLOWER]: Refusing update from remote peer 652adab41e964d87a85c0200917eb8d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:56:11.700920 2975 consensus_queue.cc:1035] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42be2478c07e4188bb6dce9855f8037f" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 41767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:56:11.701232 2979 consensus_queue.cc:1035] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "edf38a1caa55469b85c8eadd06550544" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 41019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:56:11.727470 2500 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:56:11.727752 2500 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:56:11.731196 2500 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de37c09f51624dbaa38147c7fc5ec458: Sending DeleteTablet for 3 replicas of tablet 09b5e7c6338a4f849c69595a178cfe98
I20260929 15:56:11.732426 2621 tablet_service.cc:1514] Processing DeleteTablet for tablet 09b5e7c6338a4f849c69595a178cfe98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:11 UTC) from {username='slave'} at 127.0.0.1:49308
I20260929 15:56:11.732429 2885 tablet_service.cc:1514] Processing DeleteTablet for tablet 09b5e7c6338a4f849c69595a178cfe98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:11 UTC) from {username='slave'} at 127.0.0.1:51844
I20260929 15:56:11.732466 2753 tablet_service.cc:1514] Processing DeleteTablet for tablet 09b5e7c6338a4f849c69595a178cfe98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:11 UTC) from {username='slave'} at 127.0.0.1:49200
I20260929 15:56:11.734936 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 2559
I20260929 15:56:11.744894 2994 tablet_replica.cc:330] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f: stopping tablet replica
I20260929 15:56:11.745486 2994 raft_consensus.cc:2229] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:56:11.746117 2994 raft_consensus.cc:2258] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:11.746726 2992 tablet_replica.cc:330] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4: stopping tablet replica
I20260929 15:56:11.747280 2992 raft_consensus.cc:2229] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:56:11.748100 2992 raft_consensus.cc:2258] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:11.749511 2993 tablet_replica.cc:330] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544: stopping tablet replica
I20260929 15:56:11.749920 2993 raft_consensus.cc:2229] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:56:11.750423 2993 raft_consensus.cc:2258] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:11.751590 2994 ts_tablet_manager.cc:1905] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:56:11.752128 2992 ts_tablet_manager.cc:1905] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:56:11.752234 2993 ts_tablet_manager.cc:1905] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:56:11.759229 2993 ts_tablet_manager.cc:1918] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:56:11.759482 2993 log.cc:1196] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/09b5e7c6338a4f849c69595a178cfe98
I20260929 15:56:11.759491 2994 ts_tablet_manager.cc:1918] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:56:11.759799 2994 log.cc:1196] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/09b5e7c6338a4f849c69595a178cfe98
I20260929 15:56:11.760078 2993 ts_tablet_manager.cc:1939] T 09b5e7c6338a4f849c69595a178cfe98 P edf38a1caa55469b85c8eadd06550544: Deleting consensus metadata
I20260929 15:56:11.760576 2994 ts_tablet_manager.cc:1939] T 09b5e7c6338a4f849c69595a178cfe98 P 42be2478c07e4188bb6dce9855f8037f: Deleting consensus metadata
I20260929 15:56:11.761778 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.736198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49318 (local address 127.15.196.193:41019)
0929 15:56:11.736371 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:11.736377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:11.741990 (+ 5613us) server_negotiation.cc:407] Connection header received
0929 15:56:11.742220 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:11.742226 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:11.742335 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:11.742446 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:11.745201 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.746387 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.748918 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.749630 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.749753 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:11.759710 (+ 9957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:11.759725 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:11.759749 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:11.760637 (+ 888us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:11.760702 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:11.760728 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:11.760878 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:11.761267 (+ 389us) server_negotiation.cc:299] Negotiation successful
0929 15:56:11.761398 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 15:56:11.762130 2486 catalog_manager.cc:4841] TS edf38a1caa55469b85c8eadd06550544 (127.15.196.193:41019): tablet 09b5e7c6338a4f849c69595a178cfe98 (table test-table [id=c89de71f4a184cd08080d9fd422390ac]) successfully deleted
I20260929 15:56:11.762708 2486 catalog_manager.cc:4841] TS 42be2478c07e4188bb6dce9855f8037f (127.15.196.194:41767): tablet 09b5e7c6338a4f849c69595a178cfe98 (table test-table [id=c89de71f4a184cd08080d9fd422390ac]) successfully deleted
I20260929 15:56:11.762928 2681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:11.763028 2992 ts_tablet_manager.cc:1918] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:56:11.763234 2992 log.cc:1196] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/09b5e7c6338a4f849c69595a178cfe98
I20260929 15:56:11.763801 2992 ts_tablet_manager.cc:1939] T 09b5e7c6338a4f849c69595a178cfe98 P 652adab41e964d87a85c0200917eb8d4: Deleting consensus metadata
I20260929 15:56:11.766012 2487 catalog_manager.cc:4841] TS 652adab41e964d87a85c0200917eb8d4 (127.15.196.195:46427): tablet 09b5e7c6338a4f849c69595a178cfe98 (table test-table [id=c89de71f4a184cd08080d9fd422390ac]) successfully deleted
I20260929 15:56:11.852680 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 2559
I20260929 15:56:11.881007 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 2690
I20260929 15:56:11.889747 2968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.882505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49212 (local address 127.15.196.194:41767)
0929 15:56:11.882686 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:11.882694 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:11.882718 (+ 24us) server_negotiation.cc:407] Connection header received
0929 15:56:11.882782 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:11.882788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:11.882884 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:11.882976 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:11.883600 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.884785 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.886458 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:11.886910 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:11.887049 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:11.888275 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:11.888290 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:11.888305 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:11.888922 (+ 617us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:11.888970 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:11.888990 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:11.889115 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:11.889253 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 15:56:11.889360 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 15:56:11.890519 2814 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:11.980029 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 2690
I20260929 15:56:11.997673 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 2822
I20260929 15:56:12.006394 2967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:11.998663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51854 (local address 127.15.196.195:46427)
0929 15:56:11.998842 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:11.998846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:11.998974 (+ 128us) server_negotiation.cc:407] Connection header received
0929 15:56:11.999203 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:11.999207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:11.999362 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:11.999456 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:12.000004 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:12.000797 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:12.002926 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:12.003460 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:12.003552 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:12.004923 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:12.004930 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:12.004945 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:12.005599 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:12.005654 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:12.005673 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:12.005762 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:12.005960 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 15:56:12.006035 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 15:56:12.007078 2946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:12.130452 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 2822
I20260929 15:56:12.149952 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 2469
I20260929 15:56:12.150972 2530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:12.262363 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 2469
2026-09-29T15:56:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2924 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:12 dist-test-slave-c51v krb5kdc[3002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:12 dist-test-slave-c51v krb5kdc[3002](info): set up 2 sockets
Sep 29 15:56:12 dist-test-slave-c51v krb5kdc[3002](info): commencing operation
krb5kdc: starting...
W20260929 15:56:13.389552 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.006s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:13 dist-test-slave-c51v krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:56:13.642249 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:42969
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:38955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:42969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:13.972174 3015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:13.972471 3015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:13.972690 3015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:13.979990 3015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:13.980093 3015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:13.980180 3015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:13.980288 3015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:13.980350 3015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:13.990620 3015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:38955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:42969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:42969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:13.992018 3015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:13.993361 3015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:14.005330 3020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:14.005347 3021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:14.005369 3023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:14.006107 3015 server_base.cc:1034] running on GCE node
I20260929 15:56:14.006953 3015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:14.008481 3015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:14.009737 3015 hybrid_clock.cc:648] HybridClock initialized: now 1790697374009697 us; error 51 us; skew 500 ppm
I20260929 15:56:14.010092 3015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:14 dist-test-slave-c51v krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697374, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:14.013259 3015 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:56:14.014707 3015 webserver.cc:460] Webserver started at http://127.15.196.254:39777/ using document root <none> and password file <none>
I20260929 15:56:14.015443 3015 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:14.015547 3015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:14.015801 3015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:14.018719 3015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "99ee96728e814cc1b0d0fb5332880ca4"
format_stamp: "Formatted at 2026-09-29 15:56:14 on dist-test-slave-c51v"
server_key: "465a2f17b5749f090857290a02615bad"
server_key_iv: "2044a4fa3b49a3fef6c360ed136229ed"
server_key_version: "encryptionkey@0"
I20260929 15:56:14.019510 3015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "99ee96728e814cc1b0d0fb5332880ca4"
format_stamp: "Formatted at 2026-09-29 15:56:14 on dist-test-slave-c51v"
server_key: "465a2f17b5749f090857290a02615bad"
server_key_iv: "2044a4fa3b49a3fef6c360ed136229ed"
server_key_version: "encryptionkey@0"
I20260929 15:56:14.024676 3015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260929 15:56:14.028254 3030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:14.029767 3015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:56:14.029919 3015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "99ee96728e814cc1b0d0fb5332880ca4"
format_stamp: "Formatted at 2026-09-29 15:56:14 on dist-test-slave-c51v"
server_key: "465a2f17b5749f090857290a02615bad"
server_key_iv: "2044a4fa3b49a3fef6c360ed136229ed"
server_key_version: "encryptionkey@0"
I20260929 15:56:14.030086 3015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:14.055380 3015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:14.059544 3015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:14.059796 3015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:14.078377 3015 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:42969
I20260929 15:56:14.078423 3081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:42969 every 8 connection(s)
I20260929 15:56:14.079872 3015 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:14.083577 3082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:14.086810 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 3015
I20260929 15:56:14.086998 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:14.087603 16147 external_mini_cluster.cc:1428] Setting key 6c70053d9f5eb523227d0320284b7187
I20260929 15:56:14.096400 3082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4: Bootstrap starting.
I20260929 15:56:14.100155 3082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:14.101403 3082 log.cc:824] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:14.104481 3082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4: No bootstrap required, opened a new log
Sep 29 15:56:14 dist-test-slave-c51v krb5kdc[3002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:56:14.115177 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:14.090036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.15.196.254:42969)
0929 15:56:14.091063 (+ 1027us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:14.091080 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:14.091562 (+ 482us) server_negotiation.cc:407] Connection header received
0929 15:56:14.094604 (+ 3042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:14.094635 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:14.095138 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:14.095630 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:14.097087 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:14.099133 (+ 2046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:14.100608 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:14.101670 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:14.101858 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:14.106330 (+ 4472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:14.106348 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:14.106367 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:14.106418 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:14.110530 (+ 4112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:14.111277 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:14.111306 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:14.111315 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:14.111545 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:14.111921 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:14.111926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:14.111929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:14.112661 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:14.112983 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:14.113289 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 15:56:14.113784 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":462,"threads_started":1}
I20260929 15:56:14.115392 3082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ee96728e814cc1b0d0fb5332880ca4" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 42969 } }
I20260929 15:56:14.115824 3082 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:14.115947 3082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ee96728e814cc1b0d0fb5332880ca4, State: Initialized, Role: FOLLOWER
I20260929 15:56:14.116506 3082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ee96728e814cc1b0d0fb5332880ca4" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 42969 } }
I20260929 15:56:14.116763 3082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:14.116925 3082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:14.117120 3082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:14.119522 3082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ee96728e814cc1b0d0fb5332880ca4" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 42969 } }
I20260929 15:56:14.120023 3082 leader_election.cc:302] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99ee96728e814cc1b0d0fb5332880ca4; no voters:
I20260929 15:56:14.120635 3082 leader_election.cc:288] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:14.120777 3088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:14.121479 3088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [term 1 LEADER]: Becoming Leader. State: Replica: 99ee96728e814cc1b0d0fb5332880ca4, State: Running, Role: LEADER
I20260929 15:56:14.122016 3088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ee96728e814cc1b0d0fb5332880ca4" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 42969 } }
I20260929 15:56:14.122794 3082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:14.124778 3089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99ee96728e814cc1b0d0fb5332880ca4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ee96728e814cc1b0d0fb5332880ca4" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 42969 } } }
I20260929 15:56:14.125221 3089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:14.125118 3090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99ee96728e814cc1b0d0fb5332880ca4. Latest consensus state: current_term: 1 leader_uuid: "99ee96728e814cc1b0d0fb5332880ca4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ee96728e814cc1b0d0fb5332880ca4" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 42969 } } }
I20260929 15:56:14.125417 3090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:14.126708 3093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:14.132037 3093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:14.141744 3093 catalog_manager.cc:1347] Generated new cluster ID: 7fb2f4fb80cf4e29b11df48b921dd313
I20260929 15:56:14.141881 3093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:14.162808 3093 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:14.164201 3093 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:14.171149 3093 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4: Generated new TSK 0
I20260929 15:56:14.172214 3093 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.196.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
Entry for principal kudu/127.15.196.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab.
I20260929 15:56:14.238935 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:42969
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:38955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:14.569128 3109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:14.569397 3109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:14.569561 3109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:14.576817 3109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:14.576958 3109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:14.577145 3109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:56:14.586817 3109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:38955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.193.keytab
--principal=kudu/127.15.196.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:42969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:14.588143 3109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:14.589494 3109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:14.602636 3114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:14.602749 3115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:14.602701 3117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:14.603794 3109 server_base.cc:1034] running on GCE node
I20260929 15:56:14.604478 3109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:14.605392 3109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:14.606627 3109 hybrid_clock.cc:648] HybridClock initialized: now 1790697374606580 us; error 41 us; skew 500 ppm
I20260929 15:56:14.606954 3109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:14 dist-test-slave-c51v krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697374, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:14.610363 3109 init.cc:377] Logged in from keytab as kudu/127.15.196.193@KRBTEST.COM (short username kudu)
I20260929 15:56:14.611915 3109 webserver.cc:460] Webserver started at http://127.15.196.193:43833/ using document root <none> and password file <none>
I20260929 15:56:14.612618 3109 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:14.612767 3109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:14.613093 3109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:14.616954 3109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8"
format_stamp: "Formatted at 2026-09-29 15:56:14 on dist-test-slave-c51v"
server_key: "0c90a6d90906585e5b97f5dfa01f1dc1"
server_key_iv: "113ce66c2097ede79612370639a43f73"
server_key_version: "encryptionkey@0"
I20260929 15:56:14.617722 3109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8"
format_stamp: "Formatted at 2026-09-29 15:56:14 on dist-test-slave-c51v"
server_key: "0c90a6d90906585e5b97f5dfa01f1dc1"
server_key_iv: "113ce66c2097ede79612370639a43f73"
server_key_version: "encryptionkey@0"
I20260929 15:56:14.622948 3109 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 15:56:14.626467 3124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:14.627862 3109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:14.628069 3109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8"
format_stamp: "Formatted at 2026-09-29 15:56:14 on dist-test-slave-c51v"
server_key: "0c90a6d90906585e5b97f5dfa01f1dc1"
server_key_iv: "113ce66c2097ede79612370639a43f73"
server_key_version: "encryptionkey@0"
I20260929 15:56:14.628314 3109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:14.652616 3109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:14.656791 3109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:14.657100 3109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:14.658104 3109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:14.659520 3109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:14.659623 3109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:14.659770 3109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:14.659847 3109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:14.693035 3109 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:36283
I20260929 15:56:14.693084 3237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:36283 every 8 connection(s)
I20260929 15:56:14.694404 3109 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:56:14.703961 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 3109
I20260929 15:56:14.704150 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:56:14.704603 16147 external_mini_cluster.cc:1428] Setting key 26ba8cf3232c727471bddff58a3537eb
Sep 29 15:56:14 dist-test-slave-c51v krb5kdc[3002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697374, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.193@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:56:14.714758 3240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:14.697205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:47831 (local address 127.15.196.254:42969)
0929 15:56:14.697671 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:14.697677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:14.698702 (+ 1025us) server_negotiation.cc:407] Connection header received
0929 15:56:14.700543 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:14.700547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:14.700614 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:14.700712 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:14.702151 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:14.702864 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:14.704569 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:14.705066 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:14.705214 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:14.709110 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:14.709115 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:14.709118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:14.709150 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:14.711766 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:14.712501 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:14.712505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:14.712507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:14.712624 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:14.713053 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:14.713058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:14.713068 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:14.713323 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:14.713481 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:14.714332 (+ 851us) server_negotiation.cc:299] Negotiation successful
0929 15:56:14.714510 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":227,"threads_started":1}
I20260929 15:56:14.715999 3241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:14.697805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:42969 (local address 127.15.196.193:47831)
0929 15:56:14.698533 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:14.698582 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:14.700292 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:14.700934 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:14.700941 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:14.701449 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:14.702001 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:14.702017 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:14.702996 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:14.703000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:14.704426 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:14.704438 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:14.705301 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:14.705310 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:14.705611 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:14.706303 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:14.706328 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:14.708866 (+ 2538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:14.711914 (+ 3048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:14.711924 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:14.711943 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:14.712366 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:14.712736 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:14.712742 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:14.712745 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:14.712935 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:14.713450 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:14.713458 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:14.713872 (+ 414us) client_negotiation.cc:769] Sending connection context
0929 15:56:14.714195 (+ 323us) client_negotiation.cc:240] Negotiation successful
0929 15:56:14.714799 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":261,"threads_started":1}
I20260929 15:56:14.717702 3238 heartbeater.cc:347] Connected to a master server at 127.15.196.254:42969
I20260929 15:56:14.718005 3238 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:14.718734 3238 heartbeater.cc:511] Master 127.15.196.254:42969 requested a full tablet report, sending...
I20260929 15:56:14.721093 3047 ts_manager.cc:194] Registered new tserver with Master: c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 (127.15.196.193:36283)
I20260929 15:56:14.723348 3047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.193@KRBTEST.COM'} at 127.15.196.193:47831
WARNING: no policy specified for kudu/127.15.196.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
Entry for principal kudu/127.15.196.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab.
I20260929 15:56:14.760793 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:42969
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:38955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:15.094594 3244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:15.094969 3244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:15.095080 3244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:15.102393 3244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:15.102501 3244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:15.102615 3244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:56:15.112527 3244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:38955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.194.keytab
--principal=kudu/127.15.196.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:42969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:15.113860 3244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:15.115188 3244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:15.128953 3252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:15.128948 3250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:15.128990 3249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:15.129880 3244 server_base.cc:1034] running on GCE node
I20260929 15:56:15.130503 3244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:15.131536 3244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:15.132761 3244 hybrid_clock.cc:648] HybridClock initialized: now 1790697375132710 us; error 44 us; skew 500 ppm
I20260929 15:56:15.133095 3244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:15 dist-test-slave-c51v krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697375, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:15.142632 3244 init.cc:377] Logged in from keytab as kudu/127.15.196.194@KRBTEST.COM (short username kudu)
I20260929 15:56:15.144232 3244 webserver.cc:460] Webserver started at http://127.15.196.194:38781/ using document root <none> and password file <none>
I20260929 15:56:15.144971 3244 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:15.145120 3244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:15.145450 3244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:15.148509 3244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "f0836818122e49ffb2b6f1b6d5482024"
format_stamp: "Formatted at 2026-09-29 15:56:15 on dist-test-slave-c51v"
server_key: "2cd6742de1a2c0f0e396e3643bf0dc92"
server_key_iv: "8cb79255b4fa5ccf1e6da2995a23f882"
server_key_version: "encryptionkey@0"
I20260929 15:56:15.149303 3244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "f0836818122e49ffb2b6f1b6d5482024"
format_stamp: "Formatted at 2026-09-29 15:56:15 on dist-test-slave-c51v"
server_key: "2cd6742de1a2c0f0e396e3643bf0dc92"
server_key_iv: "8cb79255b4fa5ccf1e6da2995a23f882"
server_key_version: "encryptionkey@0"
I20260929 15:56:15.154414 3244 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 15:56:15.157994 3259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:15.159320 3244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:56:15.159494 3244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "f0836818122e49ffb2b6f1b6d5482024"
format_stamp: "Formatted at 2026-09-29 15:56:15 on dist-test-slave-c51v"
server_key: "2cd6742de1a2c0f0e396e3643bf0dc92"
server_key_iv: "8cb79255b4fa5ccf1e6da2995a23f882"
server_key_version: "encryptionkey@0"
I20260929 15:56:15.159704 3244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:15.185144 3244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:15.190599 3244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:15.190922 3244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:15.191960 3244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:15.193297 3244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:15.193406 3244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:15.193562 3244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:15.193635 3244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:15.227450 3244 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:40603
I20260929 15:56:15.227535 3371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:40603 every 8 connection(s)
I20260929 15:56:15.228883 3244 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:56:15.237511 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 3244
I20260929 15:56:15.237711 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:56:15.238160 16147 external_mini_cluster.cc:1428] Setting key 06fc5e07cb88eadac9bcc94e11daf6b8
Sep 29 15:56:15 dist-test-slave-c51v krb5kdc[3002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697375, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.194@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:56:15.250680 3374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.231559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:49195 (local address 127.15.196.254:42969)
0929 15:56:15.232013 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:15.232018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:15.232914 (+ 896us) server_negotiation.cc:407] Connection header received
0929 15:56:15.234892 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:15.234898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:15.234983 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:15.235111 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:15.236718 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.237664 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.239616 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.240186 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.240315 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:15.243856 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:15.243865 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:15.243873 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:15.243919 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:15.247546 (+ 3627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:15.248274 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:15.248282 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:15.248288 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:15.248443 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:15.248852 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:15.248860 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:15.248866 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:15.249201 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:15.249315 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:15.250246 (+ 931us) server_negotiation.cc:299] Negotiation successful
0929 15:56:15.250422 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":246,"threads_started":1}
I20260929 15:56:15.252246 3375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.232081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:42969 (local address 127.15.196.194:49195)
0929 15:56:15.232757 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:15.232795 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:15.234617 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:15.235426 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:15.235436 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:15.235958 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:15.236545 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.236561 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.237850 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.237860 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:15.239473 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.239487 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.240410 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.240415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:15.240616 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:15.241352 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:15.241375 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:15.243606 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:15.247689 (+ 4083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:15.247696 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:15.247710 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:15.248142 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:15.248548 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:15.248551 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:15.248553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:15.248726 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:15.249336 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:15.249344 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:15.249814 (+ 470us) client_negotiation.cc:769] Sending connection context
0929 15:56:15.250164 (+ 350us) client_negotiation.cc:240] Negotiation successful
0929 15:56:15.250754 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":255,"threads_started":1}
I20260929 15:56:15.253738 3372 heartbeater.cc:347] Connected to a master server at 127.15.196.254:42969
I20260929 15:56:15.254084 3372 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:15.254824 3372 heartbeater.cc:511] Master 127.15.196.254:42969 requested a full tablet report, sending...
I20260929 15:56:15.256515 3047 ts_manager.cc:194] Registered new tserver with Master: f0836818122e49ffb2b6f1b6d5482024 (127.15.196.194:40603)
I20260929 15:56:15.257844 3047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.194@KRBTEST.COM'} at 127.15.196.194:49195
WARNING: no policy specified for kudu/127.15.196.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
Entry for principal kudu/127.15.196.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab.
I20260929 15:56:15.298168 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:42969
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:38955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:15.625476 3379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:15.625794 3379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:15.625959 3379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:15.633808 3379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:15.633971 3379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:15.634148 3379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:56:15.644110 3379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:38955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.195.keytab
--principal=kudu/127.15.196.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:42969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:15.645465 3379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:15.646813 3379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:15.660470 3385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:15.660913 3384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:15.660908 3387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:15.662668 3379 server_base.cc:1034] running on GCE node
I20260929 15:56:15.663396 3379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:15.664443 3379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:15.665656 3379 hybrid_clock.cc:648] HybridClock initialized: now 1790697375665594 us; error 62 us; skew 500 ppm
I20260929 15:56:15.665971 3379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:15 dist-test-slave-c51v krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697375, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:15.669103 3379 init.cc:377] Logged in from keytab as kudu/127.15.196.195@KRBTEST.COM (short username kudu)
I20260929 15:56:15.670617 3379 webserver.cc:460] Webserver started at http://127.15.196.195:39799/ using document root <none> and password file <none>
I20260929 15:56:15.671334 3379 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:15.671453 3379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:15.671720 3379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:15.674669 3379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "a845f260731d4ac88368ea0fed30f161"
format_stamp: "Formatted at 2026-09-29 15:56:15 on dist-test-slave-c51v"
server_key: "c405ea5b32d36976ea1757eeed48fac4"
server_key_iv: "953c7cc2b496aa80eca8ba9ae0b3eecb"
server_key_version: "encryptionkey@0"
I20260929 15:56:15.675467 3379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "a845f260731d4ac88368ea0fed30f161"
format_stamp: "Formatted at 2026-09-29 15:56:15 on dist-test-slave-c51v"
server_key: "c405ea5b32d36976ea1757eeed48fac4"
server_key_iv: "953c7cc2b496aa80eca8ba9ae0b3eecb"
server_key_version: "encryptionkey@0"
I20260929 15:56:15.680496 3379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 15:56:15.683943 3394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:15.685271 3379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:56:15.685397 3379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "a845f260731d4ac88368ea0fed30f161"
format_stamp: "Formatted at 2026-09-29 15:56:15 on dist-test-slave-c51v"
server_key: "c405ea5b32d36976ea1757eeed48fac4"
server_key_iv: "953c7cc2b496aa80eca8ba9ae0b3eecb"
server_key_version: "encryptionkey@0"
I20260929 15:56:15.685567 3379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:15.701021 3379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:15.704944 3379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:15.705176 3379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:15.706045 3379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:15.707342 3379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:15.707413 3379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:15.707499 3379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:15.707542 3379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:15.726598 3238 heartbeater.cc:503] Master 127.15.196.254:42969 was elected leader, sending a full tablet report...
I20260929 15:56:15.740494 3379 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:39595
I20260929 15:56:15.740531 3506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:39595 every 8 connection(s)
I20260929 15:56:15.741799 3379 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:56:15.752221 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 3379
I20260929 15:56:15.752452 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:56:15.753013 16147 external_mini_cluster.cc:1428] Setting key ee2fc07118f9435cc03d7dc4c762d0ee
Sep 29 15:56:15 dist-test-slave-c51v krb5kdc[3002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697375, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.195@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:56:15.761650 3374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.744458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:55431 (local address 127.15.196.254:42969)
0929 15:56:15.744637 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:15.744642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:15.745835 (+ 1193us) server_negotiation.cc:407] Connection header received
0929 15:56:15.747628 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:15.747633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:15.747714 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:15.747800 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:15.749244 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.749983 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.751529 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.752162 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.752334 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:15.756438 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:15.756443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:15.756446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:15.756489 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:15.758949 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:15.759700 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:15.759704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:15.759706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:15.759820 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:15.760203 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:15.760206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:15.760208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:15.760456 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:15.760589 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:15.761256 (+ 667us) server_negotiation.cc:299] Negotiation successful
0929 15:56:15.761430 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 15:56:15.762631 3509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.744979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:42969 (local address 127.15.196.195:55431)
0929 15:56:15.745670 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:15.745713 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:15.747381 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:15.748011 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:15.748019 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:15.748526 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:15.749102 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.749119 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.750117 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.750120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:15.751372 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.751383 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.752413 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.752420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:15.752690 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:15.753480 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:15.753501 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:15.756257 (+ 2756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:15.759076 (+ 2819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:15.759083 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:15.759095 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:15.759592 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:15.759921 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:15.759924 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:15.759926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:15.760107 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:15.760573 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:15.760579 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:15.760937 (+ 358us) client_negotiation.cc:769] Sending connection context
0929 15:56:15.761149 (+ 212us) client_negotiation.cc:240] Negotiation successful
0929 15:56:15.761529 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":267,"threads_started":1}
I20260929 15:56:15.764071 3507 heartbeater.cc:347] Connected to a master server at 127.15.196.254:42969
I20260929 15:56:15.764375 3507 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:15.765128 3507 heartbeater.cc:511] Master 127.15.196.254:42969 requested a full tablet report, sending...
I20260929 15:56:15.766562 3047 ts_manager.cc:194] Registered new tserver with Master: a845f260731d4ac88368ea0fed30f161 (127.15.196.195:39595)
I20260929 15:56:15.767670 3047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.196.195@KRBTEST.COM'} at 127.15.196.195:55431
I20260929 15:56:15.768678 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:56:15.785813 3374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.773377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41990 (local address 127.15.196.254:42969)
0929 15:56:15.773570 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:15.773578 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:15.774018 (+ 440us) server_negotiation.cc:407] Connection header received
0929 15:56:15.774242 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:15.774247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:15.774314 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:15.774417 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:15.775896 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.776616 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.777864 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.778271 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.778408 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:15.780798 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:15.780803 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:15.780806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:15.780837 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:15.783154 (+ 2317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:15.783823 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:15.783830 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:15.783833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:15.783966 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:15.784350 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:15.784355 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:15.784358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:15.784657 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:15.784932 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:15.785255 (+ 323us) server_negotiation.cc:299] Negotiation successful
0929 15:56:15.785479 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 15:56:15.790122 3046 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41990:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:56:15.797024 3046 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:56:15.821916 3519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.809296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:40603 (local address 127.0.0.1:46256)
0929 15:56:15.809998 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:15.810052 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:15.810252 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:15.811594 (+ 1342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:15.811605 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:15.811647 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:15.812197 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.812213 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.814945 (+ 2732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.814954 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:15.818106 (+ 3152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.818127 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.820583 (+ 2456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.820592 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:15.820917 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:15.820992 (+ 75us) client_negotiation.cc:769] Sending connection context
0929 15:56:15.821126 (+ 134us) client_negotiation.cc:240] Negotiation successful
0929 15:56:15.821367 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":276,"threads_started":1}
I20260929 15:56:15.823707 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.809558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46256 (local address 127.15.196.194:40603)
0929 15:56:15.810801 (+ 1243us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:15.810813 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:15.810859 (+ 46us) server_negotiation.cc:407] Connection header received
0929 15:56:15.810980 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:15.810991 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:15.811313 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:15.812433 (+ 1120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:15.812731 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.814728 (+ 1997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.818363 (+ 3635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.820295 (+ 1932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.822933 (+ 2638us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:15.823126 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:15.823201 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 15:56:15.823398 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1077,"thread_start_us":964,"threads_started":1}
I20260929 15:56:15.828853 3307 tablet_service.cc:1467] Processing CreateTablet for tablet 42dbd6d802ba460bb1a8335c7005b7c9 (DEFAULT_TABLE table=test-table [id=36a1d7d3ac304db5bfc8c7788ed865ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:15.830360 3307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42dbd6d802ba460bb1a8335c7005b7c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:15.831146 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.810378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:36283 (local address 127.0.0.1:33308)
0929 15:56:15.823161 (+ 12783us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:15.823177 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:15.823305 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:15.824208 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:15.824213 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:15.824246 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:15.824602 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.824614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.826434 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.826440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:15.828491 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.828505 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.830296 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.830302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:15.830502 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:15.830546 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 15:56:15.830624 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 15:56:15.830750 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12662,"thread_start_us":13384,"threads_started":1}
I20260929 15:56:15.831423 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.810935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33308 (local address 127.15.196.193:36283)
0929 15:56:15.820159 (+ 9224us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:15.820168 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:15.823433 (+ 3265us) server_negotiation.cc:407] Connection header received
0929 15:56:15.823550 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:15.823559 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:15.823816 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:15.824087 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:15.824774 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.826285 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.828684 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.830088 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.830287 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:15.830852 (+ 565us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:15.830928 (+ 76us) server_negotiation.cc:299] Negotiation successful
0929 15:56:15.831091 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9046,"thread_start_us":8906,"threads_started":1}
I20260929 15:56:15.831491 3519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.809967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:39595 (local address 127.0.0.1:44428)
0929 15:56:15.822195 (+ 12228us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:15.822219 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:15.822335 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:15.825465 (+ 3130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:15.825470 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:15.825514 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:15.825698 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.825707 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.827576 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.827582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:15.829185 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.829196 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.830689 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.830695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:15.830890 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:15.830922 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 15:56:15.830986 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 15:56:15.831136 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12103,"thread_start_us":12843,"threads_started":1}
I20260929 15:56:15.832585 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.810222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44428 (local address 127.15.196.195:39595)
0929 15:56:15.824820 (+ 14598us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:15.824829 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:15.824860 (+ 31us) server_negotiation.cc:407] Connection header received
0929 15:56:15.824948 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:15.824957 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:15.825215 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:15.825452 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:15.825863 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.827355 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.829324 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.830429 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.831869 (+ 1440us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:15.832045 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:15.832113 (+ 68us) server_negotiation.cc:299] Negotiation successful
0929 15:56:15.832272 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14408,"thread_start_us":14240,"threads_started":1}
I20260929 15:56:15.835743 3172 tablet_service.cc:1467] Processing CreateTablet for tablet 42dbd6d802ba460bb1a8335c7005b7c9 (DEFAULT_TABLE table=test-table [id=36a1d7d3ac304db5bfc8c7788ed865ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:15.836076 3442 tablet_service.cc:1467] Processing CreateTablet for tablet 42dbd6d802ba460bb1a8335c7005b7c9 (DEFAULT_TABLE table=test-table [id=36a1d7d3ac304db5bfc8c7788ed865ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:15.837183 3172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42dbd6d802ba460bb1a8335c7005b7c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:15.837378 3442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42dbd6d802ba460bb1a8335c7005b7c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:15.842734 3525 tablet_bootstrap.cc:492] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024: Bootstrap starting.
I20260929 15:56:15.846153 3525 tablet_bootstrap.cc:654] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:15.847159 3525 log.cc:824] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:15.849761 3525 tablet_bootstrap.cc:492] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024: No bootstrap required, opened a new log
I20260929 15:56:15.850070 3525 ts_tablet_manager.cc:1397] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260929 15:56:15.851956 3528 tablet_bootstrap.cc:492] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161: Bootstrap starting.
I20260929 15:56:15.855649 3528 tablet_bootstrap.cc:654] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:15.856004 3527 tablet_bootstrap.cc:492] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8: Bootstrap starting.
I20260929 15:56:15.857059 3528 log.cc:824] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:15.859503 3527 tablet_bootstrap.cc:654] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:15.860435 3527 log.cc:824] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:15.862754 3525 raft_consensus.cc:348] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0836818122e49ffb2b6f1b6d5482024" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 40603 } } peers { permanent_uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36283 } } peers { permanent_uuid: "a845f260731d4ac88368ea0fed30f161" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39595 } }
I20260929 15:56:15.863054 3527 tablet_bootstrap.cc:492] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8: No bootstrap required, opened a new log
I20260929 15:56:15.863186 3525 raft_consensus.cc:374] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:15.863338 3525 raft_consensus.cc:729] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0836818122e49ffb2b6f1b6d5482024, State: Initialized, Role: FOLLOWER
I20260929 15:56:15.863368 3527 ts_tablet_manager.cc:1397] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260929 15:56:15.864001 3525 consensus_queue.cc:260] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0836818122e49ffb2b6f1b6d5482024" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 40603 } } peers { permanent_uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36283 } } peers { permanent_uuid: "a845f260731d4ac88368ea0fed30f161" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39595 } }
I20260929 15:56:15.865134 3372 heartbeater.cc:503] Master 127.15.196.254:42969 was elected leader, sending a full tablet report...
I20260929 15:56:15.865773 3525 ts_tablet_manager.cc:1428] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260929 15:56:15.866598 3528 tablet_bootstrap.cc:492] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161: No bootstrap required, opened a new log
I20260929 15:56:15.866935 3528 ts_tablet_manager.cc:1397] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260929 15:56:15.871606 3527 raft_consensus.cc:348] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0836818122e49ffb2b6f1b6d5482024" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 40603 } } peers { permanent_uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36283 } } peers { permanent_uuid: "a845f260731d4ac88368ea0fed30f161" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39595 } }
I20260929 15:56:15.872131 3527 raft_consensus.cc:374] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:15.872229 3527 raft_consensus.cc:729] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8, State: Initialized, Role: FOLLOWER
I20260929 15:56:15.872921 3527 consensus_queue.cc:260] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0836818122e49ffb2b6f1b6d5482024" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 40603 } } peers { permanent_uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36283 } } peers { permanent_uuid: "a845f260731d4ac88368ea0fed30f161" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39595 } }
I20260929 15:56:15.874527 3527 ts_tablet_manager.cc:1428] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8: Time spent starting tablet: real 0.011s user 0.004s sys 0.006s
I20260929 15:56:15.876025 3528 raft_consensus.cc:348] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0836818122e49ffb2b6f1b6d5482024" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 40603 } } peers { permanent_uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36283 } } peers { permanent_uuid: "a845f260731d4ac88368ea0fed30f161" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39595 } }
I20260929 15:56:15.876431 3528 raft_consensus.cc:374] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:15.876535 3528 raft_consensus.cc:729] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a845f260731d4ac88368ea0fed30f161, State: Initialized, Role: FOLLOWER
I20260929 15:56:15.877120 3528 consensus_queue.cc:260] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0836818122e49ffb2b6f1b6d5482024" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 40603 } } peers { permanent_uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36283 } } peers { permanent_uuid: "a845f260731d4ac88368ea0fed30f161" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39595 } }
I20260929 15:56:15.878142 3507 heartbeater.cc:503] Master 127.15.196.254:42969 was elected leader, sending a full tablet report...
I20260929 15:56:15.878819 3528 ts_tablet_manager.cc:1428] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
W20260929 15:56:15.947232 3239 tablet.cc:2369] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:56:15.978562 3531 raft_consensus.cc:482] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:56:15.978860 3531 raft_consensus.cc:504] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0836818122e49ffb2b6f1b6d5482024" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 40603 } } peers { permanent_uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36283 } } peers { permanent_uuid: "a845f260731d4ac88368ea0fed30f161" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39595 } }
I20260929 15:56:15.980556 3531 leader_election.cc:288] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 (127.15.196.193:36283), a845f260731d4ac88368ea0fed30f161 (127.15.196.195:39595)
W20260929 15:56:15.981050 3373 tablet.cc:2369] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:56:15.988144 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.981120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:37657 (local address 127.15.196.193:36283)
0929 15:56:15.982260 (+ 1140us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:15.982270 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:15.982474 (+ 204us) server_negotiation.cc:407] Connection header received
0929 15:56:15.982781 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:15.982789 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:15.982867 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:15.982934 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:15.983783 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.984507 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.986364 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.987158 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.987377 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:15.987455 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:15.987751 (+ 296us) server_negotiation.cc:299] Negotiation successful
0929 15:56:15.987875 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":946}
I20260929 15:56:15.988399 3536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.980734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:36283 (local address 127.15.196.194:37657)
0929 15:56:15.982357 (+ 1623us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:15.982383 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:15.982604 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:15.983130 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:15.983137 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:15.983166 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:15.983626 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.983641 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.984659 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.984663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:15.986184 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.986196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.987402 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.987409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:15.987584 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:15.987610 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 15:56:15.987724 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 15:56:15.987869 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1483,"mutex_wait_us":1,"thread_start_us":1904,"threads_started":1}
I20260929 15:56:15.988412 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.981482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:49123 (local address 127.15.196.195:39595)
0929 15:56:15.981656 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:15.981662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:15.983024 (+ 1362us) server_negotiation.cc:407] Connection header received
0929 15:56:15.983220 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:15.983228 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:15.983331 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:15.983426 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:15.984042 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.984774 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.986496 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.987237 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.987515 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:15.987586 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:15.987907 (+ 321us) server_negotiation.cc:299] Negotiation successful
0929 15:56:15.988035 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 15:56:15.988427 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.980903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:39595 (local address 127.15.196.194:49123)
0929 15:56:15.982916 (+ 2013us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:15.982930 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:15.983072 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:15.983473 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:15.983478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:15.983504 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:15.983893 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.983901 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.984940 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.984945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:15.986313 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.986325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.987535 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.987541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:15.987732 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:15.987755 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 15:56:15.987877 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 15:56:15.988001 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1908,"thread_start_us":1082,"threads_started":1}
I20260929 15:56:15.989621 3462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42dbd6d802ba460bb1a8335c7005b7c9" candidate_uuid: "f0836818122e49ffb2b6f1b6d5482024" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a845f260731d4ac88368ea0fed30f161" is_pre_election: true
I20260929 15:56:15.989675 3192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42dbd6d802ba460bb1a8335c7005b7c9" candidate_uuid: "f0836818122e49ffb2b6f1b6d5482024" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8" is_pre_election: true
I20260929 15:56:15.990185 3462 raft_consensus.cc:2454] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0836818122e49ffb2b6f1b6d5482024 in term 0.
I20260929 15:56:15.990298 3192 raft_consensus.cc:2454] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0836818122e49ffb2b6f1b6d5482024 in term 0.
I20260929 15:56:15.990526 3533 raft_consensus.cc:482] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:56:15.990763 3533 raft_consensus.cc:504] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0836818122e49ffb2b6f1b6d5482024" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 40603 } } peers { permanent_uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36283 } } peers { permanent_uuid: "a845f260731d4ac88368ea0fed30f161" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39595 } }
I20260929 15:56:15.990921 3261 leader_election.cc:302] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a845f260731d4ac88368ea0fed30f161, f0836818122e49ffb2b6f1b6d5482024; no voters:
I20260929 15:56:15.991349 3531 raft_consensus.cc:2781] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:56:15.991510 3531 raft_consensus.cc:482] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:56:15.991649 3531 raft_consensus.cc:3037] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:15.992635 3533 leader_election.cc:288] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0836818122e49ffb2b6f1b6d5482024 (127.15.196.194:40603), a845f260731d4ac88368ea0fed30f161 (127.15.196.195:39595)
W20260929 15:56:15.993387 3508 tablet.cc:2369] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:56:15.994704 3531 raft_consensus.cc:504] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0836818122e49ffb2b6f1b6d5482024" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 40603 } } peers { permanent_uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36283 } } peers { permanent_uuid: "a845f260731d4ac88368ea0fed30f161" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39595 } }
I20260929 15:56:15.995559 3531 leader_election.cc:288] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [CANDIDATE]: Term 1 election: Requested vote from peers c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 (127.15.196.193:36283), a845f260731d4ac88368ea0fed30f161 (127.15.196.195:39595)
I20260929 15:56:15.995975 3462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42dbd6d802ba460bb1a8335c7005b7c9" candidate_uuid: "f0836818122e49ffb2b6f1b6d5482024" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a845f260731d4ac88368ea0fed30f161"
I20260929 15:56:15.996187 3462 raft_consensus.cc:3037] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:15.996160 3192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42dbd6d802ba460bb1a8335c7005b7c9" candidate_uuid: "f0836818122e49ffb2b6f1b6d5482024" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8"
I20260929 15:56:15.996372 3192 raft_consensus.cc:3037] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:15.999547 3462 raft_consensus.cc:2454] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0836818122e49ffb2b6f1b6d5482024 in term 1.
I20260929 15:56:16.000226 3192 raft_consensus.cc:2454] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0836818122e49ffb2b6f1b6d5482024 in term 1.
I20260929 15:56:16.000620 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.993182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:34179 (local address 127.15.196.194:40603)
0929 15:56:15.993365 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:15.993371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:15.993424 (+ 53us) server_negotiation.cc:407] Connection header received
0929 15:56:15.993619 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:15.993624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:15.993692 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:15.993795 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:15.994349 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.995430 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.998253 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.999544 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.999680 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:15.999751 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:16.000236 (+ 485us) server_negotiation.cc:299] Negotiation successful
0929 15:56:16.000369 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 15:56:16.000743 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.992809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:40603 (local address 127.15.196.193:34179)
0929 15:56:15.993341 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:15.993361 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:15.993499 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:15.993830 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:15.993834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:15.993866 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:15.994201 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.994215 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.995589 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.995595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:15.998051 (+ 2456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.998068 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.999958 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.999962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:16.000076 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:16.000098 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 15:56:16.000214 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 15:56:16.000343 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":1,"thread_start_us":1633,"threads_started":1}
I20260929 15:56:16.000646 3261 leader_election.cc:302] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a845f260731d4ac88368ea0fed30f161, f0836818122e49ffb2b6f1b6d5482024; no voters:
I20260929 15:56:16.001173 3531 raft_consensus.cc:2781] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:16.001855 3327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42dbd6d802ba460bb1a8335c7005b7c9" candidate_uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0836818122e49ffb2b6f1b6d5482024" is_pre_election: true
I20260929 15:56:16.002310 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.992822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:39595 (local address 127.15.196.193:45245)
0929 15:56:15.994886 (+ 2064us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:15.994904 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:15.995016 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:15.995336 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:15.995340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:15.995363 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:15.995850 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.995864 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.997391 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.997415 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:15.999612 (+ 2197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:15.999626 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:16.001464 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:16.001473 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:16.001645 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:16.001680 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 15:56:16.001779 (+ 99us) client_negotiation.cc:240] Negotiation successful
0929 15:56:16.001936 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1919,"mutex_wait_us":1413,"thread_start_us":527,"threads_started":1}
I20260929 15:56:16.002410 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:15.993020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:45245 (local address 127.15.196.195:39595)
0929 15:56:15.994753 (+ 1733us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:15.994758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:15.994965 (+ 207us) server_negotiation.cc:407] Connection header received
0929 15:56:15.995140 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:15.995145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:15.995216 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:15.995324 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:15.996064 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:15.997193 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:15.999786 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:16.001089 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:16.001211 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:16.001277 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:16.001841 (+ 564us) server_negotiation.cc:299] Negotiation successful
0929 15:56:16.002032 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1617}
I20260929 15:56:16.002954 3462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42dbd6d802ba460bb1a8335c7005b7c9" candidate_uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a845f260731d4ac88368ea0fed30f161" is_pre_election: true
I20260929 15:56:16.003058 3531 raft_consensus.cc:686] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [term 1 LEADER]: Becoming Leader. State: Replica: f0836818122e49ffb2b6f1b6d5482024, State: Running, Role: LEADER
I20260929 15:56:16.003312 3462 raft_consensus.cc:2379] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 in current term 1: Already voted for candidate f0836818122e49ffb2b6f1b6d5482024 in this term.
I20260929 15:56:16.004027 3531 consensus_queue.cc:237] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0836818122e49ffb2b6f1b6d5482024" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 40603 } } peers { permanent_uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36283 } } peers { permanent_uuid: "a845f260731d4ac88368ea0fed30f161" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39595 } }
I20260929 15:56:16.007786 3127 leader_election.cc:302] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8; no voters: a845f260731d4ac88368ea0fed30f161, f0836818122e49ffb2b6f1b6d5482024
I20260929 15:56:16.008188 3533 raft_consensus.cc:2726] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260929 15:56:16.009065 3046 catalog_manager.cc:5495] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0836818122e49ffb2b6f1b6d5482024 (127.15.196.194). New cstate: current_term: 1 leader_uuid: "f0836818122e49ffb2b6f1b6d5482024" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0836818122e49ffb2b6f1b6d5482024" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 40603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a845f260731d4ac88368ea0fed30f161" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39595 } health_report { overall_health: UNKNOWN } } }
I20260929 15:56:16.031438 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:16.026233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46264 (local address 127.15.196.194:40603)
0929 15:56:16.026424 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:16.026428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:16.026464 (+ 36us) server_negotiation.cc:407] Connection header received
0929 15:56:16.026568 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:16.026571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:16.026654 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:16.026775 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:16.027099 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:16.027890 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:16.029295 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:16.029697 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:16.029794 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:16.030184 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:16.030684 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:16.030804 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:16.030931 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 15:56:16.031042 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 15:56:16.039556 3462 raft_consensus.cc:1261] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161 [term 1 FOLLOWER]: Refusing update from remote peer f0836818122e49ffb2b6f1b6d5482024: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:56:16.039605 3192 raft_consensus.cc:1261] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 [term 1 FOLLOWER]: Refusing update from remote peer f0836818122e49ffb2b6f1b6d5482024: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:56:16.040496 3539 consensus_queue.cc:1035] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a845f260731d4ac88368ea0fed30f161" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 39595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:56:16.040819 3531 consensus_queue.cc:1035] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 36283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:56:16.052156 3542 mvcc.cc:205] Tried to move back new op lower bound from 7334696452248788992 to 7334696452116226048. Current Snapshot: MvccSnapshot[applied={T|T < 7334696452248788992}]
I20260929 15:56:16.067855 3047 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:56:16.068168 3047 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:56:16.072021 3047 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99ee96728e814cc1b0d0fb5332880ca4: Sending DeleteTablet for 3 replicas of tablet 42dbd6d802ba460bb1a8335c7005b7c9
I20260929 15:56:16.073078 3307 tablet_service.cc:1514] Processing DeleteTablet for tablet 42dbd6d802ba460bb1a8335c7005b7c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:16 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:46256
I20260929 15:56:16.073378 3172 tablet_service.cc:1514] Processing DeleteTablet for tablet 42dbd6d802ba460bb1a8335c7005b7c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:16 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:33308
I20260929 15:56:16.073483 3442 tablet_service.cc:1514] Processing DeleteTablet for tablet 42dbd6d802ba460bb1a8335c7005b7c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:16 UTC) from {username='kudu', principal='kudu/127.15.196.254@KRBTEST.COM'} at 127.0.0.1:44428
I20260929 15:56:16.079430 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 3109
I20260929 15:56:16.088364 3552 tablet_replica.cc:330] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024: stopping tablet replica
I20260929 15:56:16.088867 3552 raft_consensus.cc:2229] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:56:16.089820 3552 raft_consensus.cc:2258] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:16.092317 3552 ts_tablet_manager.cc:1905] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:56:16.099927 3552 ts_tablet_manager.cc:1918] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:56:16.100096 3552 log.cc:1196] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/42dbd6d802ba460bb1a8335c7005b7c9
I20260929 15:56:16.100660 3552 ts_tablet_manager.cc:1939] T 42dbd6d802ba460bb1a8335c7005b7c9 P f0836818122e49ffb2b6f1b6d5482024: Deleting consensus metadata
I20260929 15:56:16.101327 3553 tablet_replica.cc:330] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8: stopping tablet replica
I20260929 15:56:16.103950 3033 catalog_manager.cc:4841] TS f0836818122e49ffb2b6f1b6d5482024 (127.15.196.194:40603): tablet 42dbd6d802ba460bb1a8335c7005b7c9 (table test-table [id=36a1d7d3ac304db5bfc8c7788ed865ff]) successfully deleted
I20260929 15:56:16.107496 3553 raft_consensus.cc:2229] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:56:16.108032 3553 raft_consensus.cc:2258] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:16.109529 3554 tablet_replica.cc:330] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161: stopping tablet replica
I20260929 15:56:16.109956 3554 raft_consensus.cc:2229] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:56:16.110421 3554 raft_consensus.cc:2258] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:16.111091 3553 ts_tablet_manager.cc:1905] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 29 15:56:16 dist-test-slave-c51v krb5kdc[3002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.193@KRBTEST.COM
I20260929 15:56:16.118448 3554 ts_tablet_manager.cc:1905] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:56:16.118489 3553 ts_tablet_manager.cc:1918] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:56:16.118834 3553 log.cc:1196] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-0/wal/wals/42dbd6d802ba460bb1a8335c7005b7c9
I20260929 15:56:16.119494 3553 ts_tablet_manager.cc:1939] T 42dbd6d802ba460bb1a8335c7005b7c9 P c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8: Deleting consensus metadata
I20260929 15:56:16.121495 3033 catalog_manager.cc:4841] TS c5c5f8f7ca7e4c7fb5bdc19cfb08b9c8 (127.15.196.193:36283): tablet 42dbd6d802ba460bb1a8335c7005b7c9 (table test-table [id=36a1d7d3ac304db5bfc8c7788ed865ff]) successfully deleted
I20260929 15:56:16.124224 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:16.080430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33324 (local address 127.15.196.193:36283)
0929 15:56:16.080596 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:16.080602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:16.097757 (+ 17155us) server_negotiation.cc:407] Connection header received
0929 15:56:16.098191 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:16.098197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:16.098280 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:16.098363 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:16.101862 (+ 3499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:16.103430 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:16.106789 (+ 3359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:16.107566 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:16.107708 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:16.115048 (+ 7340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:16.115062 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:16.115080 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:16.115127 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:16.120157 (+ 5030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:16.121104 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:16.121115 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:16.121119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:16.121769 (+ 650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:16.122294 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:16.122300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:16.122303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:16.122838 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:16.123130 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:16.123597 (+ 467us) server_negotiation.cc:299] Negotiation successful
0929 15:56:16.123855 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 15:56:16.124933 3233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:16.131403 3554 ts_tablet_manager.cc:1918] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:56:16.131767 3554 log.cc:1196] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/42dbd6d802ba460bb1a8335c7005b7c9
I20260929 15:56:16.132440 3554 ts_tablet_manager.cc:1939] T 42dbd6d802ba460bb1a8335c7005b7c9 P a845f260731d4ac88368ea0fed30f161: Deleting consensus metadata
I20260929 15:56:16.135044 3032 catalog_manager.cc:4841] TS a845f260731d4ac88368ea0fed30f161 (127.15.196.195:39595): tablet 42dbd6d802ba460bb1a8335c7005b7c9 (table test-table [id=36a1d7d3ac304db5bfc8c7788ed865ff]) successfully deleted
I20260929 15:56:16.224270 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 3109
I20260929 15:56:16.251627 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 3244
Sep 29 15:56:16 dist-test-slave-c51v krb5kdc[3002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.194@KRBTEST.COM
I20260929 15:56:16.273703 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:16.252758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46270 (local address 127.15.196.194:40603)
0929 15:56:16.252979 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:16.252985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:16.253005 (+ 20us) server_negotiation.cc:407] Connection header received
0929 15:56:16.253067 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:16.253073 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:16.253152 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:16.253272 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:16.254959 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:16.256362 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:16.258420 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:16.259178 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:16.259337 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:16.265948 (+ 6611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:16.265961 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:16.265980 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:16.266019 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:16.270227 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:16.271160 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:16.271170 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:16.271174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:16.271353 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:16.271806 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:16.271812 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:16.271815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:16.272353 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:16.272638 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:16.272935 (+ 297us) server_negotiation.cc:299] Negotiation successful
0929 15:56:16.273332 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 15:56:16.274521 3367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:16.395051 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 3244
I20260929 15:56:16.414348 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 3379
Sep 29 15:56:16 dist-test-slave-c51v krb5kdc[3002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.195@KRBTEST.COM
I20260929 15:56:16.429103 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:16.415113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44444 (local address 127.15.196.195:39595)
0929 15:56:16.415348 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:16.415355 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:16.415382 (+ 27us) server_negotiation.cc:407] Connection header received
0929 15:56:16.415485 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:16.415489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:16.415564 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:16.415653 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:16.416915 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:16.417584 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:16.418837 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:16.419220 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:16.419383 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:16.423631 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:16.423646 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:16.423662 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:16.423693 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:16.426257 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:16.426890 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:16.426897 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:16.426900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:16.427018 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:16.427668 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:16.427672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:16.427674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:16.428150 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:16.428382 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:16.428649 (+ 267us) server_negotiation.cc:299] Negotiation successful
0929 15:56:16.428804 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 15:56:16.429816 3502 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:16.518560 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 3379
I20260929 15:56:16.535434 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 3015
I20260929 15:56:16.536468 3077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:16.646127 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 3015
2026-09-29T15:56:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4385 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-29T15:56:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:16Z Disabled control of system clock
I20260929 15:56:16.723917 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:43927
--webserver_interface=127.15.196.254
--webserver_port=0
--builtin_ntp_servers=127.15.196.212:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:43927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 15:56:17.054554 3563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:17.054880 3563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:17.055016 3563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:17.062480 3563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:17.062584 3563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:17.062621 3563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:17.062682 3563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:17.062744 3563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:17.071979 3563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:43927
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:43927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:17.073182 3563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:17.074883 3563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:17.087033 3568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:17.087072 3570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:17.087217 3572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:17.088140 3563 server_base.cc:1034] running on GCE node
I20260929 15:56:17.088994 3563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:17.090489 3563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:17.091720 3563 hybrid_clock.cc:648] HybridClock initialized: now 1790697377091681 us; error 51 us; skew 500 ppm
I20260929 15:56:17.092090 3563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:56:17.094331 3563 webserver.cc:460] Webserver started at http://127.15.196.254:38679/ using document root <none> and password file <none>
I20260929 15:56:17.095026 3563 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:17.095160 3563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:17.095594 3563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:17.098575 3563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "5a1e5071fc2344159178c74af0e6c8bc"
format_stamp: "Formatted at 2026-09-29 15:56:17 on dist-test-slave-c51v"
server_key: "ea987468bf2f5fd8e019a6665d6bdaa0"
server_key_iv: "9c82d28d6bcad3edf0284caf7223d4f6"
server_key_version: "encryptionkey@0"
I20260929 15:56:17.099463 3563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "5a1e5071fc2344159178c74af0e6c8bc"
format_stamp: "Formatted at 2026-09-29 15:56:17 on dist-test-slave-c51v"
server_key: "ea987468bf2f5fd8e019a6665d6bdaa0"
server_key_iv: "9c82d28d6bcad3edf0284caf7223d4f6"
server_key_version: "encryptionkey@0"
I20260929 15:56:17.104594 3563 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 15:56:17.108084 3578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:17.109395 3563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:56:17.109591 3563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "5a1e5071fc2344159178c74af0e6c8bc"
format_stamp: "Formatted at 2026-09-29 15:56:17 on dist-test-slave-c51v"
server_key: "ea987468bf2f5fd8e019a6665d6bdaa0"
server_key_iv: "9c82d28d6bcad3edf0284caf7223d4f6"
server_key_version: "encryptionkey@0"
I20260929 15:56:17.109803 3563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:17.129266 3563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:17.130270 3563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:17.130587 3563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:17.149284 3563 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:43927
I20260929 15:56:17.149304 3629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:43927 every 8 connection(s)
I20260929 15:56:17.150678 3563 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:17.154330 3630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:17.156852 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 3563
I20260929 15:56:17.157058 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:17.157491 16147 external_mini_cluster.cc:1428] Setting key c0b25e42950575f2ca338c4c7741f08a
I20260929 15:56:17.165359 3630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc: Bootstrap starting.
I20260929 15:56:17.168313 3630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:17.169380 3630 log.cc:824] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:17.170725 3633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:17.159360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34104 (local address 127.15.196.254:43927)
0929 15:56:17.160272 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:17.160283 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:17.160332 (+ 49us) server_negotiation.cc:407] Connection header received
0929 15:56:17.161963 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:17.161983 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:17.162435 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:17.162826 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:17.163423 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:17.164665 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:17.166033 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:17.166724 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:17.166939 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:17.168099 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:17.168106 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:17.168118 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:17.168689 (+ 571us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:17.168780 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:17.168820 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:17.168948 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:17.169166 (+ 218us) server_negotiation.cc:299] Negotiation successful
0929 15:56:17.169457 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":321,"threads_started":1}
I20260929 15:56:17.172498 3630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc: No bootstrap required, opened a new log
I20260929 15:56:17.179774 3630 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1e5071fc2344159178c74af0e6c8bc" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43927 } }
I20260929 15:56:17.180154 3630 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:17.180261 3630 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a1e5071fc2344159178c74af0e6c8bc, State: Initialized, Role: FOLLOWER
I20260929 15:56:17.180789 3630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1e5071fc2344159178c74af0e6c8bc" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43927 } }
I20260929 15:56:17.181047 3630 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:17.181191 3630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:17.181353 3630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:17.183554 3630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1e5071fc2344159178c74af0e6c8bc" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43927 } }
I20260929 15:56:17.184031 3630 leader_election.cc:302] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a1e5071fc2344159178c74af0e6c8bc; no voters:
I20260929 15:56:17.184605 3630 leader_election.cc:288] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:17.184753 3635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:17.185477 3635 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [term 1 LEADER]: Becoming Leader. State: Replica: 5a1e5071fc2344159178c74af0e6c8bc, State: Running, Role: LEADER
I20260929 15:56:17.186005 3635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1e5071fc2344159178c74af0e6c8bc" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43927 } }
I20260929 15:56:17.186779 3630 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:17.189237 3637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a1e5071fc2344159178c74af0e6c8bc. Latest consensus state: current_term: 1 leader_uuid: "5a1e5071fc2344159178c74af0e6c8bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1e5071fc2344159178c74af0e6c8bc" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43927 } } }
I20260929 15:56:17.189214 3636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a1e5071fc2344159178c74af0e6c8bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1e5071fc2344159178c74af0e6c8bc" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 43927 } } }
I20260929 15:56:17.189672 3636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:17.189669 3637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:17.191896 3642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:17.197826 3642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:17.207561 3642 catalog_manager.cc:1347] Generated new cluster ID: e278a9c7892c4515aaf58979afdb6fa5
I20260929 15:56:17.207681 3642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:17.225082 3642 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:17.226420 3642 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:17.233211 3642 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc: Generated new TSK 0
I20260929 15:56:17.234062 3642 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:17.243877 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.193:0
--local_ip_for_outbound_sockets=127.15.196.193
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:43927
--builtin_ntp_servers=127.15.196.212:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 15:56:17.577854 3655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:17.578141 3655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:17.578234 3655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:17.585299 3655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:17.585397 3655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:17.585500 3655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.193
I20260929 15:56:17.594792 3655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.196.193
--webserver_port=0
--tserver_master_addrs=127.15.196.254:43927
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.193
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:17.596084 3655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:17.597877 3655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:17.611142 3661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:17.611410 3660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:17.611121 3663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:17.612149 3655 server_base.cc:1034] running on GCE node
I20260929 15:56:17.612722 3655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:17.613816 3655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:17.615015 3655 hybrid_clock.cc:648] HybridClock initialized: now 1790697377614945 us; error 55 us; skew 500 ppm
I20260929 15:56:17.615413 3655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:56:17.617715 3655 webserver.cc:460] Webserver started at http://127.15.196.193:40395/ using document root <none> and password file <none>
I20260929 15:56:17.618451 3655 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:17.618564 3655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:17.618902 3655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:17.621838 3655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/data/instance:
uuid: "464f2f6407274314802e0912f2241da6"
format_stamp: "Formatted at 2026-09-29 15:56:17 on dist-test-slave-c51v"
server_key: "dbfb415cf3baddb9ef8a8d7ec978130c"
server_key_iv: "f42369aa364f61548921f2c5105cc9b3"
server_key_version: "encryptionkey@0"
I20260929 15:56:17.622620 3655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance:
uuid: "464f2f6407274314802e0912f2241da6"
format_stamp: "Formatted at 2026-09-29 15:56:17 on dist-test-slave-c51v"
server_key: "dbfb415cf3baddb9ef8a8d7ec978130c"
server_key_iv: "f42369aa364f61548921f2c5105cc9b3"
server_key_version: "encryptionkey@0"
I20260929 15:56:17.627904 3655 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260929 15:56:17.631399 3669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:17.632720 3655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:56:17.632926 3655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/wal
uuid: "464f2f6407274314802e0912f2241da6"
format_stamp: "Formatted at 2026-09-29 15:56:17 on dist-test-slave-c51v"
server_key: "dbfb415cf3baddb9ef8a8d7ec978130c"
server_key_iv: "f42369aa364f61548921f2c5105cc9b3"
server_key_version: "encryptionkey@0"
I20260929 15:56:17.633138 3655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:17.646087 3655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:17.646899 3655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:17.647200 3655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:17.648242 3655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:17.649677 3655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:17.649806 3655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:17.649940 3655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:17.650030 3655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:17.682627 3655 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.193:33981
I20260929 15:56:17.682678 3782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.193:33981 every 8 connection(s)
I20260929 15:56:17.684047 3655 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/data/info.pb
I20260929 15:56:17.689070 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 3655
I20260929 15:56:17.689251 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-0/wal/instance
I20260929 15:56:17.689682 16147 external_mini_cluster.cc:1428] Setting key f1d16b76d990f793c5a0a754e3523926
I20260929 15:56:17.692289 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.194:0
--local_ip_for_outbound_sockets=127.15.196.194
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:43927
--builtin_ntp_servers=127.15.196.212:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 15:56:17.697213 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:17.686554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.193:43747 (local address 127.15.196.254:43927)
0929 15:56:17.687028 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:17.687032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:17.688463 (+ 1431us) server_negotiation.cc:407] Connection header received
0929 15:56:17.690563 (+ 2100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:17.690568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:17.690665 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:17.690787 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:17.692017 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:17.692820 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:17.694422 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:17.694826 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:17.694955 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:17.696385 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:17.696390 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:17.696393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:17.696412 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:17.696454 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:17.696466 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:17.696560 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:17.696787 (+ 227us) server_negotiation.cc:299] Negotiation successful
0929 15:56:17.696889 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":264,"threads_started":1}
I20260929 15:56:17.698282 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:17.687175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:43927 (local address 127.15.196.193:43747)
0929 15:56:17.688296 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:17.688353 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:17.690260 (+ 1907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:17.691028 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:17.691036 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:17.691153 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:17.691854 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:17.691871 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:17.692969 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:17.692974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:17.694269 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:17.694281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:17.695063 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:17.695071 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:17.695360 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:17.696079 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:56:17.696081 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:17.696120 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:56:17.696195 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:17.696559 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:17.696571 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:17.696581 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 15:56:17.696766 (+ 185us) client_negotiation.cc:240] Negotiation successful
0929 15:56:17.697149 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":293,"threads_started":1}
I20260929 15:56:17.699936 3783 heartbeater.cc:347] Connected to a master server at 127.15.196.254:43927
I20260929 15:56:17.700237 3783 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:17.700968 3783 heartbeater.cc:511] Master 127.15.196.254:43927 requested a full tablet report, sending...
I20260929 15:56:17.702955 3595 ts_manager.cc:194] Registered new tserver with Master: 464f2f6407274314802e0912f2241da6 (127.15.196.193:33981)
I20260929 15:56:17.704598 3595 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.196.193:43747
W20260929 15:56:18.027477 3787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:18.027791 3787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:18.027971 3787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:18.035516 3787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:18.035686 3787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:18.035830 3787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.194
I20260929 15:56:18.045568 3787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.196.194
--webserver_port=0
--tserver_master_addrs=127.15.196.254:43927
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.194
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:18.046963 3787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:18.048806 3787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:18.062837 3792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:18.062908 3793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:18.062904 3795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:18.069042 3787 server_base.cc:1034] running on GCE node
I20260929 15:56:18.069694 3787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:18.070675 3787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:18.071903 3787 hybrid_clock.cc:648] HybridClock initialized: now 1790697378071844 us; error 63 us; skew 500 ppm
I20260929 15:56:18.072221 3787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:56:18.074537 3787 webserver.cc:460] Webserver started at http://127.15.196.194:42915/ using document root <none> and password file <none>
I20260929 15:56:18.075246 3787 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:18.075392 3787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:18.075711 3787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:18.078728 3787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/data/instance:
uuid: "a051ccf20fcf4ae0977ac687b51ea894"
format_stamp: "Formatted at 2026-09-29 15:56:18 on dist-test-slave-c51v"
server_key: "9df8bfcd34940bd0ae60519f217def55"
server_key_iv: "efda5e09fe40a1ee454f2c0b1d179122"
server_key_version: "encryptionkey@0"
I20260929 15:56:18.079718 3787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance:
uuid: "a051ccf20fcf4ae0977ac687b51ea894"
format_stamp: "Formatted at 2026-09-29 15:56:18 on dist-test-slave-c51v"
server_key: "9df8bfcd34940bd0ae60519f217def55"
server_key_iv: "efda5e09fe40a1ee454f2c0b1d179122"
server_key_version: "encryptionkey@0"
I20260929 15:56:18.087477 3787 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260929 15:56:18.091202 3801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:18.092648 3787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:18.092828 3787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/wal
uuid: "a051ccf20fcf4ae0977ac687b51ea894"
format_stamp: "Formatted at 2026-09-29 15:56:18 on dist-test-slave-c51v"
server_key: "9df8bfcd34940bd0ae60519f217def55"
server_key_iv: "efda5e09fe40a1ee454f2c0b1d179122"
server_key_version: "encryptionkey@0"
I20260929 15:56:18.093103 3787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:18.107995 3787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:18.108762 3787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:18.109019 3787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:18.109934 3787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:18.111342 3787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:18.111441 3787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:18.111526 3787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:18.111579 3787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:18.145304 3787 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.194:44541
I20260929 15:56:18.145339 3914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.194:44541 every 8 connection(s)
I20260929 15:56:18.146785 3787 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/data/info.pb
I20260929 15:56:18.147797 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 3787
I20260929 15:56:18.147989 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/wal/instance
I20260929 15:56:18.148521 16147 external_mini_cluster.cc:1428] Setting key b7d295e71ebe21fa844a7bb50b57c57f
I20260929 15:56:18.152271 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.196.195:0
--local_ip_for_outbound_sockets=127.15.196.195
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:43927
--builtin_ntp_servers=127.15.196.212:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 15:56:18.163125 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.150550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:35355 (local address 127.15.196.254:43927)
0929 15:56:18.150745 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:18.150751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:18.152497 (+ 1746us) server_negotiation.cc:407] Connection header received
0929 15:56:18.155067 (+ 2570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:18.155075 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:18.155165 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:18.155269 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:18.156679 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.157825 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.160143 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.160658 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.160757 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.162285 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:18.162290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:18.162293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:18.162310 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:18.162340 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:18.162352 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:18.162442 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:18.162684 (+ 242us) server_negotiation.cc:299] Negotiation successful
0929 15:56:18.162783 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260929 15:56:18.164175 3917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.151156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:43927 (local address 127.15.196.194:35355)
0929 15:56:18.152338 (+ 1182us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:18.152391 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:18.154737 (+ 2346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:18.155590 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:18.155601 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:18.155721 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:18.156476 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:18.156501 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.157995 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.158000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:18.159974 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:18.159989 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.160892 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.160899 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:18.161190 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.161964 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:56:18.161967 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:18.162011 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:56:18.162089 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:18.162456 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:18.162468 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:18.162483 (+ 15us) client_negotiation.cc:769] Sending connection context
0929 15:56:18.162658 (+ 175us) client_negotiation.cc:240] Negotiation successful
0929 15:56:18.163067 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":348,"threads_started":1}
I20260929 15:56:18.165691 3915 heartbeater.cc:347] Connected to a master server at 127.15.196.254:43927
I20260929 15:56:18.166028 3915 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:18.166783 3915 heartbeater.cc:511] Master 127.15.196.254:43927 requested a full tablet report, sending...
I20260929 15:56:18.168207 3595 ts_manager.cc:194] Registered new tserver with Master: a051ccf20fcf4ae0977ac687b51ea894 (127.15.196.194:44541)
I20260929 15:56:18.169059 3595 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.196.194:35355
W20260929 15:56:18.488513 3918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:18.488794 3918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:18.488951 3918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:18.496260 3918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:18.496410 3918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:18.496584 3918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.196.195
I20260929 15:56:18.505983 3918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.196.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.196.195
--webserver_port=0
--tserver_master_addrs=127.15.196.254:43927
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.196.195
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:18.507361 3918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:18.509114 3918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:18.522840 3924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:18.522807 3923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:18.523546 3926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:18.523698 3918 server_base.cc:1034] running on GCE node
I20260929 15:56:18.524399 3918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:18.525394 3918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:18.526630 3918 hybrid_clock.cc:648] HybridClock initialized: now 1790697378526587 us; error 56 us; skew 500 ppm
I20260929 15:56:18.526973 3918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 15:56:18.529367 3918 webserver.cc:460] Webserver started at http://127.15.196.195:45143/ using document root <none> and password file <none>
I20260929 15:56:18.530089 3918 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:18.530222 3918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:18.530545 3918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:18.533597 3918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/data/instance:
uuid: "59f7c349101d4912b28bf018f9c941b4"
format_stamp: "Formatted at 2026-09-29 15:56:18 on dist-test-slave-c51v"
server_key: "d2f05c0120a6925c33ab7f377d32aece"
server_key_iv: "f7c580b5c4a07a2b65e298b0b6d9d32a"
server_key_version: "encryptionkey@0"
I20260929 15:56:18.534399 3918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance:
uuid: "59f7c349101d4912b28bf018f9c941b4"
format_stamp: "Formatted at 2026-09-29 15:56:18 on dist-test-slave-c51v"
server_key: "d2f05c0120a6925c33ab7f377d32aece"
server_key_iv: "f7c580b5c4a07a2b65e298b0b6d9d32a"
server_key_version: "encryptionkey@0"
I20260929 15:56:18.539712 3918 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 15:56:18.543177 3932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:18.544533 3918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:56:18.544715 3918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/wal
uuid: "59f7c349101d4912b28bf018f9c941b4"
format_stamp: "Formatted at 2026-09-29 15:56:18 on dist-test-slave-c51v"
server_key: "d2f05c0120a6925c33ab7f377d32aece"
server_key_iv: "f7c580b5c4a07a2b65e298b0b6d9d32a"
server_key_version: "encryptionkey@0"
I20260929 15:56:18.544929 3918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:18.557741 3918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:18.558578 3918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:18.558884 3918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:18.559913 3918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 15:56:18.561336 3918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 15:56:18.561465 3918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:18.561594 3918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 15:56:18.561678 3918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:18.594789 3918 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.195:34437
I20260929 15:56:18.594838 4045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.195:34437 every 8 connection(s)
I20260929 15:56:18.596529 3918 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/data/info.pb
I20260929 15:56:18.603374 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 3918
I20260929 15:56:18.603574 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/wal/instance
I20260929 15:56:18.604019 16147 external_mini_cluster.cc:1428] Setting key f8da762b0a8cb8761981551d571884e4
I20260929 15:56:18.609644 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.599441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.195:43491 (local address 127.15.196.254:43927)
0929 15:56:18.599704 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:18.599710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:18.601236 (+ 1526us) server_negotiation.cc:407] Connection header received
0929 15:56:18.603089 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:18.603095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:18.603204 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:18.603356 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:18.604544 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.605400 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.606928 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.607412 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.607545 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.608859 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:18.608864 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:18.608868 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:18.608885 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:18.608916 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:18.608927 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:18.609025 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:18.609238 (+ 213us) server_negotiation.cc:299] Negotiation successful
0929 15:56:18.609332 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 15:56:18.610670 4048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.600180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:43927 (local address 127.15.196.195:43491)
0929 15:56:18.601069 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:18.601105 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:18.602686 (+ 1581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:18.603625 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:18.603640 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:18.603749 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:18.604375 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:18.604395 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.605559 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.605564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:18.606781 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:18.606793 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.607629 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.607636 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:18.607867 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.608543 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 15:56:18.608545 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:18.608592 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 15:56:18.608666 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:18.609023 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:18.609035 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:18.609045 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 15:56:18.609229 (+ 184us) client_negotiation.cc:240] Negotiation successful
0929 15:56:18.609634 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":275,"threads_started":1}
I20260929 15:56:18.611950 4046 heartbeater.cc:347] Connected to a master server at 127.15.196.254:43927
I20260929 15:56:18.612241 4046 heartbeater.cc:464] Registering TS with master...
I20260929 15:56:18.612973 4046 heartbeater.cc:511] Master 127.15.196.254:43927 requested a full tablet report, sending...
I20260929 15:56:18.614355 3595 ts_manager.cc:194] Registered new tserver with Master: 59f7c349101d4912b28bf018f9c941b4 (127.15.196.195:34437)
I20260929 15:56:18.615238 3595 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.196.195:43491
I20260929 15:56:18.618773 16147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 15:56:18.628273 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.622755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34110 (local address 127.15.196.254:43927)
0929 15:56:18.623031 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:18.623037 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:18.623365 (+ 328us) server_negotiation.cc:407] Connection header received
0929 15:56:18.623530 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:18.623534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:18.623606 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:18.623689 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:18.624154 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.624830 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.626104 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.626480 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.626566 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.627593 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:18.627598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:18.627601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:18.627618 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:18.627653 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:18.627665 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:18.627736 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:18.627860 (+ 124us) server_negotiation.cc:299] Negotiation successful
0929 15:56:18.627930 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 15:56:18.631852 3595 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 15:56:18.638849 3595 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 15:56:18.679437 4059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.650818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47142 (local address 127.15.196.193:33981)
0929 15:56:18.652061 (+ 1243us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:18.652073 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:18.666585 (+ 14512us) server_negotiation.cc:407] Connection header received
0929 15:56:18.668610 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:18.668623 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:18.668985 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:18.669219 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:18.670361 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.672374 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.676776 (+ 4402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.678211 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.678387 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.678645 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:18.678892 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 15:56:18.679055 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1071,"thread_start_us":1242,"threads_started":1}
I20260929 15:56:18.679498 4062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.650636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:33981 (local address 127.0.0.1:47142)
0929 15:56:18.666390 (+ 15754us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:18.666451 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:18.668273 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:18.669573 (+ 1300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:18.669586 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:18.669629 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:18.670130 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:18.670145 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.672604 (+ 2459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.672613 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:18.676565 (+ 3952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:18.676579 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.678425 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.678431 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:18.678627 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.678679 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 15:56:18.678839 (+ 160us) client_negotiation.cc:240] Negotiation successful
0929 15:56:18.679028 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15395,"thread_start_us":25473,"threads_started":1}
I20260929 15:56:18.683835 3717 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb1761c08d748e19ca5b2ed316fbc0c (DEFAULT_TABLE table=test-table [id=68a1160fde8c40f9be76b8b3222e4fac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:18.685038 4058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.651215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.194:44541 (local address 127.0.0.1:57236)
0929 15:56:18.676257 (+ 25042us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:18.676278 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:18.676415 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:18.677041 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:18.677046 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:18.677070 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:18.677407 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:18.677417 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.679171 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.679177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:18.681588 (+ 2411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:18.681603 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.683832 (+ 2229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.683839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:18.684031 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.684402 (+ 371us) client_negotiation.cc:769] Sending connection context
0929 15:56:18.684489 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 15:56:18.684633 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24928,"mutex_wait_us":47,"spinlock_wait_cycles":5632,"thread_start_us":14661,"threads_started":1}
I20260929 15:56:18.685267 3717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb1761c08d748e19ca5b2ed316fbc0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:18.685688 4060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.651152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:34437 (local address 127.0.0.1:36862)
0929 15:56:18.676234 (+ 25082us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:18.676262 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:18.676415 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:18.677378 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:18.677383 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:18.677411 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:18.677804 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:18.677814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.680134 (+ 2320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.680139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:18.682295 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:18.682309 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.684566 (+ 2257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.684573 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:18.684737 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.684768 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 15:56:18.684848 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 15:56:18.684982 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24906,"mutex_wait_us":28,"spinlock_wait_cycles":10624,"thread_start_us":24710,"threads_started":1}
I20260929 15:56:18.685837 4063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.651924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57236 (local address 127.15.196.194:44541)
0929 15:56:18.675752 (+ 23828us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:18.675762 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:18.676399 (+ 637us) server_negotiation.cc:407] Connection header received
0929 15:56:18.676639 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:18.676645 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:18.676853 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:18.677064 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:18.677569 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.679011 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.681838 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.683341 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.683497 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.683708 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:18.685326 (+ 1618us) server_negotiation.cc:299] Negotiation successful
0929 15:56:18.685504 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23687,"thread_start_us":23586,"threads_started":1}
I20260929 15:56:18.687716 4061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.651140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36862 (local address 127.15.196.195:34437)
0929 15:56:18.672804 (+ 21664us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:18.672816 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:18.676483 (+ 3667us) server_negotiation.cc:407] Connection header received
0929 15:56:18.676601 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:18.676611 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:18.676906 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:18.677096 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:18.678020 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.679969 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.682511 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.683938 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.684117 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.684330 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:18.687139 (+ 2809us) server_negotiation.cc:299] Negotiation successful
0929 15:56:18.687338 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21416,"thread_start_us":21694,"threads_started":1}
I20260929 15:56:18.689674 3849 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb1761c08d748e19ca5b2ed316fbc0c (DEFAULT_TABLE table=test-table [id=68a1160fde8c40f9be76b8b3222e4fac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:18.691097 3849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb1761c08d748e19ca5b2ed316fbc0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:18.691615 3980 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb1761c08d748e19ca5b2ed316fbc0c (DEFAULT_TABLE table=test-table [id=68a1160fde8c40f9be76b8b3222e4fac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 15:56:18.693142 3980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb1761c08d748e19ca5b2ed316fbc0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:18.700842 4064 tablet_bootstrap.cc:492] T 7cb1761c08d748e19ca5b2ed316fbc0c P 464f2f6407274314802e0912f2241da6: Bootstrap starting.
I20260929 15:56:18.703696 4065 tablet_bootstrap.cc:492] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894: Bootstrap starting.
I20260929 15:56:18.705317 4064 tablet_bootstrap.cc:654] T 7cb1761c08d748e19ca5b2ed316fbc0c P 464f2f6407274314802e0912f2241da6: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:18.706310 4064 log.cc:824] T 7cb1761c08d748e19ca5b2ed316fbc0c P 464f2f6407274314802e0912f2241da6: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:18.707432 4065 tablet_bootstrap.cc:654] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:18.707535 3783 heartbeater.cc:503] Master 127.15.196.254:43927 was elected leader, sending a full tablet report...
I20260929 15:56:18.708927 4065 log.cc:824] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:18.710379 4064 tablet_bootstrap.cc:492] T 7cb1761c08d748e19ca5b2ed316fbc0c P 464f2f6407274314802e0912f2241da6: No bootstrap required, opened a new log
I20260929 15:56:18.710647 4064 ts_tablet_manager.cc:1397] T 7cb1761c08d748e19ca5b2ed316fbc0c P 464f2f6407274314802e0912f2241da6: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260929 15:56:18.713176 4066 tablet_bootstrap.cc:492] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4: Bootstrap starting.
I20260929 15:56:18.716053 4065 tablet_bootstrap.cc:492] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894: No bootstrap required, opened a new log
I20260929 15:56:18.716315 4065 ts_tablet_manager.cc:1397] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.006s
I20260929 15:56:18.716868 4066 tablet_bootstrap.cc:654] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:18.718063 4066 log.cc:824] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:18.719971 4064 raft_consensus.cc:348] T 7cb1761c08d748e19ca5b2ed316fbc0c P 464f2f6407274314802e0912f2241da6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f2f6407274314802e0912f2241da6" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33981 } } peers { permanent_uuid: "59f7c349101d4912b28bf018f9c941b4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 34437 } } peers { permanent_uuid: "a051ccf20fcf4ae0977ac687b51ea894" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44541 } }
I20260929 15:56:18.720391 4064 raft_consensus.cc:374] T 7cb1761c08d748e19ca5b2ed316fbc0c P 464f2f6407274314802e0912f2241da6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:18.720500 4064 raft_consensus.cc:729] T 7cb1761c08d748e19ca5b2ed316fbc0c P 464f2f6407274314802e0912f2241da6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 464f2f6407274314802e0912f2241da6, State: Initialized, Role: FOLLOWER
I20260929 15:56:18.721179 4064 consensus_queue.cc:260] T 7cb1761c08d748e19ca5b2ed316fbc0c P 464f2f6407274314802e0912f2241da6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f2f6407274314802e0912f2241da6" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33981 } } peers { permanent_uuid: "59f7c349101d4912b28bf018f9c941b4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 34437 } } peers { permanent_uuid: "a051ccf20fcf4ae0977ac687b51ea894" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44541 } }
I20260929 15:56:18.726938 4064 ts_tablet_manager.cc:1428] T 7cb1761c08d748e19ca5b2ed316fbc0c P 464f2f6407274314802e0912f2241da6: Time spent starting tablet: real 0.016s user 0.007s sys 0.009s
I20260929 15:56:18.727594 4066 tablet_bootstrap.cc:492] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4: No bootstrap required, opened a new log
I20260929 15:56:18.727897 4066 ts_tablet_manager.cc:1397] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260929 15:56:18.728704 4065 raft_consensus.cc:348] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f2f6407274314802e0912f2241da6" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33981 } } peers { permanent_uuid: "59f7c349101d4912b28bf018f9c941b4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 34437 } } peers { permanent_uuid: "a051ccf20fcf4ae0977ac687b51ea894" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44541 } }
I20260929 15:56:18.729166 4065 raft_consensus.cc:374] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:18.729279 4065 raft_consensus.cc:729] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a051ccf20fcf4ae0977ac687b51ea894, State: Initialized, Role: FOLLOWER
I20260929 15:56:18.730276 4065 consensus_queue.cc:260] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f2f6407274314802e0912f2241da6" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33981 } } peers { permanent_uuid: "59f7c349101d4912b28bf018f9c941b4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 34437 } } peers { permanent_uuid: "a051ccf20fcf4ae0977ac687b51ea894" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44541 } }
I20260929 15:56:18.730666 4071 raft_consensus.cc:482] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 15:56:18.730955 4071 raft_consensus.cc:504] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f2f6407274314802e0912f2241da6" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33981 } } peers { permanent_uuid: "59f7c349101d4912b28bf018f9c941b4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 34437 } } peers { permanent_uuid: "a051ccf20fcf4ae0977ac687b51ea894" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44541 } }
I20260929 15:56:18.731655 3915 heartbeater.cc:503] Master 127.15.196.254:43927 was elected leader, sending a full tablet report...
I20260929 15:56:18.732239 4065 ts_tablet_manager.cc:1428] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894: Time spent starting tablet: real 0.016s user 0.014s sys 0.002s
I20260929 15:56:18.739444 4071 leader_election.cc:288] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 464f2f6407274314802e0912f2241da6 (127.15.196.193:33981), 59f7c349101d4912b28bf018f9c941b4 (127.15.196.195:34437)
I20260929 15:56:18.739796 4066 raft_consensus.cc:348] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f2f6407274314802e0912f2241da6" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33981 } } peers { permanent_uuid: "59f7c349101d4912b28bf018f9c941b4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 34437 } } peers { permanent_uuid: "a051ccf20fcf4ae0977ac687b51ea894" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44541 } }
I20260929 15:56:18.740304 4066 raft_consensus.cc:374] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:18.740425 4066 raft_consensus.cc:729] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59f7c349101d4912b28bf018f9c941b4, State: Initialized, Role: FOLLOWER
I20260929 15:56:18.741210 4066 consensus_queue.cc:260] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f2f6407274314802e0912f2241da6" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33981 } } peers { permanent_uuid: "59f7c349101d4912b28bf018f9c941b4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 34437 } } peers { permanent_uuid: "a051ccf20fcf4ae0977ac687b51ea894" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44541 } }
I20260929 15:56:18.742326 4046 heartbeater.cc:503] Master 127.15.196.254:43927 was elected leader, sending a full tablet report...
I20260929 15:56:18.743021 4066 ts_tablet_manager.cc:1428] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
I20260929 15:56:18.747972 4074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.733210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:33981 (local address 127.15.196.194:59691)
0929 15:56:18.740050 (+ 6840us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:18.740073 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:18.740256 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:18.740865 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:18.740868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:18.740915 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:18.741311 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:18.741325 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.743353 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.743359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:18.745958 (+ 2599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:18.745972 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.747157 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.747164 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:18.747370 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.747402 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 15:56:18.747489 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 15:56:18.747615 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6689,"mutex_wait_us":172,"thread_start_us":7342,"threads_started":1}
I20260929 15:56:18.747983 4061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.733446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:45003 (local address 127.15.196.195:34437)
0929 15:56:18.733640 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:18.733647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:18.740928 (+ 7281us) server_negotiation.cc:407] Connection header received
0929 15:56:18.741087 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:18.741094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:18.741168 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:18.741275 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:18.741964 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.743122 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.746170 (+ 3048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.746965 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.747155 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.747221 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:18.747602 (+ 381us) server_negotiation.cc:299] Negotiation successful
0929 15:56:18.747724 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 15:56:18.748093 4073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.733221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.195:34437 (local address 127.15.196.194:45003)
0929 15:56:18.740816 (+ 7595us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:18.740834 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:18.740954 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:18.741326 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:18.741331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:18.741356 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 15:56:18.741778 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:18.741796 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.743335 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.743342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:18.745958 (+ 2616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:18.745972 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.747176 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.747182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:18.747429 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.747468 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 15:56:18.747579 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 15:56:18.747703 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7466,"mutex_wait_us":597,"thread_start_us":6085,"threads_started":1}
I20260929 15:56:18.748580 4059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.733828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.196.194:59691 (local address 127.15.196.193:33981)
0929 15:56:18.734014 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:18.734020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:18.740184 (+ 6164us) server_negotiation.cc:407] Connection header received
0929 15:56:18.740467 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:18.740476 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:18.740582 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:18.740673 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 15:56:18.741506 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.742556 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.746205 (+ 3649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.746953 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.747102 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.747186 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:18.748202 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0929 15:56:18.748328 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 15:56:18.749034 4000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb1761c08d748e19ca5b2ed316fbc0c" candidate_uuid: "a051ccf20fcf4ae0977ac687b51ea894" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59f7c349101d4912b28bf018f9c941b4" is_pre_election: true
I20260929 15:56:18.749599 4000 raft_consensus.cc:2454] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a051ccf20fcf4ae0977ac687b51ea894 in term 0.
I20260929 15:56:18.749719 3738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb1761c08d748e19ca5b2ed316fbc0c" candidate_uuid: "a051ccf20fcf4ae0977ac687b51ea894" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464f2f6407274314802e0912f2241da6" is_pre_election: true
I20260929 15:56:18.750263 3738 raft_consensus.cc:2454] T 7cb1761c08d748e19ca5b2ed316fbc0c P 464f2f6407274314802e0912f2241da6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a051ccf20fcf4ae0977ac687b51ea894 in term 0.
I20260929 15:56:18.750283 3805 leader_election.cc:302] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59f7c349101d4912b28bf018f9c941b4, a051ccf20fcf4ae0977ac687b51ea894; no voters:
I20260929 15:56:18.750712 4071 raft_consensus.cc:2781] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 15:56:18.750867 4071 raft_consensus.cc:482] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 15:56:18.750993 4071 raft_consensus.cc:3037] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:18.753171 4071 raft_consensus.cc:504] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f2f6407274314802e0912f2241da6" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33981 } } peers { permanent_uuid: "59f7c349101d4912b28bf018f9c941b4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 34437 } } peers { permanent_uuid: "a051ccf20fcf4ae0977ac687b51ea894" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44541 } }
I20260929 15:56:18.753834 4071 leader_election.cc:288] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [CANDIDATE]: Term 1 election: Requested vote from peers 464f2f6407274314802e0912f2241da6 (127.15.196.193:33981), 59f7c349101d4912b28bf018f9c941b4 (127.15.196.195:34437)
I20260929 15:56:18.754222 3738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb1761c08d748e19ca5b2ed316fbc0c" candidate_uuid: "a051ccf20fcf4ae0977ac687b51ea894" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464f2f6407274314802e0912f2241da6"
I20260929 15:56:18.754420 3738 raft_consensus.cc:3037] T 7cb1761c08d748e19ca5b2ed316fbc0c P 464f2f6407274314802e0912f2241da6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:18.754503 4000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb1761c08d748e19ca5b2ed316fbc0c" candidate_uuid: "a051ccf20fcf4ae0977ac687b51ea894" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59f7c349101d4912b28bf018f9c941b4"
I20260929 15:56:18.754695 4000 raft_consensus.cc:3037] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:18.756774 3738 raft_consensus.cc:2454] T 7cb1761c08d748e19ca5b2ed316fbc0c P 464f2f6407274314802e0912f2241da6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a051ccf20fcf4ae0977ac687b51ea894 in term 1.
I20260929 15:56:18.757004 4000 raft_consensus.cc:2454] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a051ccf20fcf4ae0977ac687b51ea894 in term 1.
I20260929 15:56:18.757198 3802 leader_election.cc:302] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 464f2f6407274314802e0912f2241da6, a051ccf20fcf4ae0977ac687b51ea894; no voters:
I20260929 15:56:18.757500 4071 raft_consensus.cc:2781] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:18.757647 4071 raft_consensus.cc:686] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [term 1 LEADER]: Becoming Leader. State: Replica: a051ccf20fcf4ae0977ac687b51ea894, State: Running, Role: LEADER
I20260929 15:56:18.758086 4071 consensus_queue.cc:237] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f2f6407274314802e0912f2241da6" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33981 } } peers { permanent_uuid: "59f7c349101d4912b28bf018f9c941b4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 34437 } } peers { permanent_uuid: "a051ccf20fcf4ae0977ac687b51ea894" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44541 } }
I20260929 15:56:18.761978 3595 catalog_manager.cc:5495] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 reported cstate change: term changed from 0 to 1, leader changed from <none> to a051ccf20fcf4ae0977ac687b51ea894 (127.15.196.194). New cstate: current_term: 1 leader_uuid: "a051ccf20fcf4ae0977ac687b51ea894" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f2f6407274314802e0912f2241da6" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59f7c349101d4912b28bf018f9c941b4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 34437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a051ccf20fcf4ae0977ac687b51ea894" member_type: VOTER last_known_addr { host: "127.15.196.194" port: 44541 } health_report { overall_health: HEALTHY } } }
I20260929 15:56:18.789706 4063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.784671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57242 (local address 127.15.196.194:44541)
0929 15:56:18.784935 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:18.784939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:18.784958 (+ 19us) server_negotiation.cc:407] Connection header received
0929 15:56:18.785102 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:18.785106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:18.785191 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:18.785310 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:18.785610 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.786298 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.787674 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.788111 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.788205 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.788640 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:18.789163 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:18.789281 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:18.789341 (+ 60us) server_negotiation.cc:299] Negotiation successful
0929 15:56:18.789403 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260929 15:56:18.798002 3738 raft_consensus.cc:1261] T 7cb1761c08d748e19ca5b2ed316fbc0c P 464f2f6407274314802e0912f2241da6 [term 1 FOLLOWER]: Refusing update from remote peer a051ccf20fcf4ae0977ac687b51ea894: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:56:18.797988 4000 raft_consensus.cc:1261] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4 [term 1 FOLLOWER]: Refusing update from remote peer a051ccf20fcf4ae0977ac687b51ea894: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:56:18.799147 4072 consensus_queue.cc:1035] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59f7c349101d4912b28bf018f9c941b4" member_type: VOTER last_known_addr { host: "127.15.196.195" port: 34437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:56:18.799528 4071 consensus_queue.cc:1035] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [LEADER]: Connected to new peer: Peer: permanent_uuid: "464f2f6407274314802e0912f2241da6" member_type: VOTER last_known_addr { host: "127.15.196.193" port: 33981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:56:18.809746 4078 mvcc.cc:205] Tried to move back new op lower bound from 7334696463546793984 to 7334696463397281792. Current Snapshot: MvccSnapshot[applied={T|T < 7334696463546793984}]
I20260929 15:56:18.811038 4080 mvcc.cc:205] Tried to move back new op lower bound from 7334696463546793984 to 7334696463397281792. Current Snapshot: MvccSnapshot[applied={T|T < 7334696463546793984}]
I20260929 15:56:18.830523 3595 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:56:18.830797 3595 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 15:56:18.834303 3595 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a1e5071fc2344159178c74af0e6c8bc: Sending DeleteTablet for 3 replicas of tablet 7cb1761c08d748e19ca5b2ed316fbc0c
I20260929 15:56:18.835596 3980 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb1761c08d748e19ca5b2ed316fbc0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:18 UTC) from {username='slave'} at 127.0.0.1:36862
I20260929 15:56:18.835685 3849 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb1761c08d748e19ca5b2ed316fbc0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:18 UTC) from {username='slave'} at 127.0.0.1:57236
I20260929 15:56:18.836124 3717 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb1761c08d748e19ca5b2ed316fbc0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 15:56:18 UTC) from {username='slave'} at 127.0.0.1:47142
I20260929 15:56:18.836670 4089 tablet_replica.cc:330] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894: stopping tablet replica
I20260929 15:56:18.837100 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 3655
I20260929 15:56:18.837260 4089 raft_consensus.cc:2229] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [term 1 LEADER]: Raft consensus shutting down.
I20260929 15:56:18.838441 4089 raft_consensus.cc:2258] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:18.840960 4089 ts_tablet_manager.cc:1905] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:56:18.849848 4089 ts_tablet_manager.cc:1918] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:56:18.850116 4089 log.cc:1196] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-1/wal/wals/7cb1761c08d748e19ca5b2ed316fbc0c
I20260929 15:56:18.850844 4089 ts_tablet_manager.cc:1939] T 7cb1761c08d748e19ca5b2ed316fbc0c P a051ccf20fcf4ae0977ac687b51ea894: Deleting consensus metadata
I20260929 15:56:18.851038 4088 tablet_replica.cc:330] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4: stopping tablet replica
I20260929 15:56:18.851711 4088 raft_consensus.cc:2229] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:56:18.852342 4088 raft_consensus.cc:2258] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 15:56:18.853137 3580 catalog_manager.cc:4841] TS a051ccf20fcf4ae0977ac687b51ea894 (127.15.196.194:44541): tablet 7cb1761c08d748e19ca5b2ed316fbc0c (table test-table [id=68a1160fde8c40f9be76b8b3222e4fac]) successfully deleted
I20260929 15:56:18.855449 4088 ts_tablet_manager.cc:1905] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 15:56:18.861363 4059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:18.838865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47150 (local address 127.15.196.193:33981)
0929 15:56:18.839048 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:18.839056 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:18.851252 (+ 12196us) server_negotiation.cc:407] Connection header received
0929 15:56:18.852965 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:18.852970 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:18.853098 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:18.853212 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:18.853907 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.855180 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.857199 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:18.857934 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:18.858035 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:18.859599 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:18.859611 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:18.859629 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:18.860429 (+ 800us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:18.860491 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:18.860517 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:18.860603 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:18.860845 (+ 242us) server_negotiation.cc:299] Negotiation successful
0929 15:56:18.860962 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 15:56:18.862883 3778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:18.867420 4088 ts_tablet_manager.cc:1918] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 15:56:18.867605 4088 log.cc:1196] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4: Deleting WAL directory at /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697059131395-16147-0/minicluster-data/ts-2/wal/wals/7cb1761c08d748e19ca5b2ed316fbc0c
I20260929 15:56:18.868076 4088 ts_tablet_manager.cc:1939] T 7cb1761c08d748e19ca5b2ed316fbc0c P 59f7c349101d4912b28bf018f9c941b4: Deleting consensus metadata
I20260929 15:56:18.869697 3582 catalog_manager.cc:4841] TS 59f7c349101d4912b28bf018f9c941b4 (127.15.196.195:34437): tablet 7cb1761c08d748e19ca5b2ed316fbc0c (table test-table [id=68a1160fde8c40f9be76b8b3222e4fac]) successfully deleted
I20260929 15:56:18.984786 4090 tablet_replica.cc:330] T 7cb1761c08d748e19ca5b2ed316fbc0c P 464f2f6407274314802e0912f2241da6: stopping tablet replica
I20260929 15:56:18.985611 4090 raft_consensus.cc:2229] T 7cb1761c08d748e19ca5b2ed316fbc0c P 464f2f6407274314802e0912f2241da6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 15:56:18.985747 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 3655
W20260929 15:56:19.003662 3579 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.196.193:33981 (error 108)
I20260929 15:56:19.004395 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 3787
I20260929 15:56:19.006013 4060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:19.005536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:33981 (local address 127.0.0.1:47166)
0929 15:56:19.005702 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:19.005826 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:33981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260929 15:56:19.006242 3579 catalog_manager.cc:4584] TS 464f2f6407274314802e0912f2241da6 (127.15.196.193:33981): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7cb1761c08d748e19ca5b2ed316fbc0c: Network error: Client connection negotiation failed: client connection to 127.15.196.193:33981: connect: Connection refused (error 111)
I20260929 15:56:19.011685 4063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:19.005189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57248 (local address 127.15.196.194:44541)
0929 15:56:19.005372 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:19.005383 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:19.005440 (+ 57us) server_negotiation.cc:407] Connection header received
0929 15:56:19.005689 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:19.005696 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:19.005797 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:19.005911 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:19.006385 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:19.007195 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:19.008534 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:19.008983 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:19.009122 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:19.010171 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:19.010180 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:19.010197 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:19.010798 (+ 601us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:19.010859 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:19.010878 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:19.011005 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:19.011140 (+ 135us) server_negotiation.cc:299] Negotiation successful
0929 15:56:19.011243 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 15:56:19.012357 3910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:19.060014 4060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:19.059549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:33981 (local address 127.0.0.1:47172)
0929 15:56:19.059750 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:19.059837 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:33981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260929 15:56:19.061044 4060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:19.060693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:33981 (local address 127.0.0.1:47188)
0929 15:56:19.060844 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:19.060933 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:33981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260929 15:56:19.121582 4060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:19.121064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:33981 (local address 127.0.0.1:47194)
0929 15:56:19.121303 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:19.121402 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:33981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260929 15:56:19.122579 4060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:19.122234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:33981 (local address 127.0.0.1:47196)
0929 15:56:19.122401 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:19.122467 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:33981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 15:56:19.125569 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 3787
I20260929 15:56:19.145056 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 3918
I20260929 15:56:19.152485 4061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:19.145837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36866 (local address 127.15.196.195:34437)
0929 15:56:19.146016 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:19.146023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:19.146119 (+ 96us) server_negotiation.cc:407] Connection header received
0929 15:56:19.146338 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:19.146345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:19.146428 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:19.146534 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:19.146927 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:19.147892 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:19.149538 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:19.149925 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:19.150021 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:19.151075 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:19.151084 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:19.151099 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:19.151768 (+ 669us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:19.151815 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:19.151837 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:19.151921 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:19.152069 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 15:56:19.152153 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 15:56:19.153213 4041 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:19.203212 4060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:19.202551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:33981 (local address 127.0.0.1:47204)
0929 15:56:19.202856 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:19.202973 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:33981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260929 15:56:19.204582 4060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:19.204052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.193:33981 (local address 127.0.0.1:47214)
0929 15:56:19.204322 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:19.204424 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.193:33981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
I20260929 15:56:19.244081 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 3918
I20260929 15:56:19.262810 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 3563
I20260929 15:56:19.264016 3625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:19.373416 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 3563
2026-09-29T15:56:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2726 ms)
[----------] 6 tests from EncryptionPolicyTest (21440 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:19 dist-test-slave-c51v krb5kdc[4099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:19 dist-test-slave-c51v krb5kdc[4099](info): set up 2 sockets
Sep 29 15:56:19 dist-test-slave-c51v krb5kdc[4099](info): commencing operation
krb5kdc: starting...
W20260929 15:56:20.481918 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.006s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:20 dist-test-slave-c51v krb5kdc[4099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 15:56:20.738569 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41427
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:21.067721 4112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:21.068080 4112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:21.068238 4112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:21.075518 4112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:21.075668 4112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:21.075780 4112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:21.075893 4112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:21.075969 4112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:21.086097 4112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41427
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:21.087520 4112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:21.089371 4112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:21.102770 4117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:21.102732 4120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:21.102700 4118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:21.103619 4112 server_base.cc:1034] running on GCE node
I20260929 15:56:21.104490 4112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:21.105943 4112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:21.107173 4112 hybrid_clock.cc:648] HybridClock initialized: now 1790697381107142 us; error 58 us; skew 500 ppm
I20260929 15:56:21.107550 4112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:21 dist-test-slave-c51v krb5kdc[4099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697381, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:21.118243 4112 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 15:56:21.119951 4112 webserver.cc:460] Webserver started at http://127.0.0.1:41363/ using document root <none> and password file <none>
I20260929 15:56:21.120726 4112 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:21.120878 4112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:21.121202 4112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:21.124481 4112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "c37f24c9e29b431ea96f28a9cc0887d9"
format_stamp: "Formatted at 2026-09-29 15:56:21 on dist-test-slave-c51v"
server_key: "689c2e622414c0b55837e9463796f461"
server_key_iv: "24b2017c397960d3a7db6ff27a74d7c7"
server_key_version: "encryptionkey@0"
I20260929 15:56:21.125278 4112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "c37f24c9e29b431ea96f28a9cc0887d9"
format_stamp: "Formatted at 2026-09-29 15:56:21 on dist-test-slave-c51v"
server_key: "689c2e622414c0b55837e9463796f461"
server_key_iv: "24b2017c397960d3a7db6ff27a74d7c7"
server_key_version: "encryptionkey@0"
I20260929 15:56:21.130667 4112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 15:56:21.134187 4127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:21.135521 4112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:21.135704 4112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "c37f24c9e29b431ea96f28a9cc0887d9"
format_stamp: "Formatted at 2026-09-29 15:56:21 on dist-test-slave-c51v"
server_key: "689c2e622414c0b55837e9463796f461"
server_key_iv: "24b2017c397960d3a7db6ff27a74d7c7"
server_key_version: "encryptionkey@0"
I20260929 15:56:21.135926 4112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:21.153422 4112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:21.157260 4112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:21.157552 4112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:21.175905 4112 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41427
I20260929 15:56:21.175937 4179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41427 every 8 connection(s)
I20260929 15:56:21.177258 4112 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:21.180902 4180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:21.181631 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 4112
I20260929 15:56:21.181855 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:21.182507 16147 external_mini_cluster.cc:1428] Setting key 42b604480e3eea9f721dc36c1dbcde4b
I20260929 15:56:21.194558 4180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9: Bootstrap starting.
Sep 29 15:56:21 dist-test-slave-c51v krb5kdc[4099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 15:56:21.198347 4180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:21.199524 4180 log.cc:824] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:21.202461 4180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9: No bootstrap required, opened a new log
I20260929 15:56:21.206055 4183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:21.184408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43793 (local address 127.0.0.1:41427)
0929 15:56:21.185259 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:21.185270 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:21.185315 (+ 45us) server_negotiation.cc:407] Connection header received
0929 15:56:21.186930 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:21.186954 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:21.187402 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:21.187721 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:21.189519 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:21.190796 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:21.192705 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:21.193364 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:21.193489 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:21.198301 (+ 4812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:21.198310 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:21.198323 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:21.198368 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:21.201659 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:21.202478 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:21.202484 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:21.202490 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:21.202705 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:21.203178 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:21.203187 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:21.203190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:21.203775 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:21.204142 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:21.204661 (+ 519us) server_negotiation.cc:299] Negotiation successful
0929 15:56:21.205005 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":374,"threads_started":1}
I20260929 15:56:21.210407 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:21.211167 4180 raft_consensus.cc:348] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37f24c9e29b431ea96f28a9cc0887d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41427 } }
I20260929 15:56:21.211615 4180 raft_consensus.cc:374] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:21.211710 4180 raft_consensus.cc:729] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37f24c9e29b431ea96f28a9cc0887d9, State: Initialized, Role: FOLLOWER
I20260929 15:56:21.212394 4180 consensus_queue.cc:260] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37f24c9e29b431ea96f28a9cc0887d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41427 } }
I20260929 15:56:21.212652 4180 raft_consensus.cc:388] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:21.212785 4180 raft_consensus.cc:482] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:21.212965 4180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:21.216625 4180 raft_consensus.cc:504] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37f24c9e29b431ea96f28a9cc0887d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41427 } }
I20260929 15:56:21.217288 4180 leader_election.cc:302] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c37f24c9e29b431ea96f28a9cc0887d9; no voters:
I20260929 15:56:21.218042 4180 leader_election.cc:288] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:21.218184 4190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:21.219552 4190 raft_consensus.cc:686] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [term 1 LEADER]: Becoming Leader. State: Replica: c37f24c9e29b431ea96f28a9cc0887d9, State: Running, Role: LEADER
I20260929 15:56:21.220254 4190 consensus_queue.cc:237] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37f24c9e29b431ea96f28a9cc0887d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41427 } }
I20260929 15:56:21.220530 4180 sys_catalog.cc:564] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:21.227805 4191 sys_catalog.cc:455] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c37f24c9e29b431ea96f28a9cc0887d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37f24c9e29b431ea96f28a9cc0887d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41427 } } }
I20260929 15:56:21.229090 4191 sys_catalog.cc:458] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:21.228318 4192 sys_catalog.cc:455] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c37f24c9e29b431ea96f28a9cc0887d9. Latest consensus state: current_term: 1 leader_uuid: "c37f24c9e29b431ea96f28a9cc0887d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37f24c9e29b431ea96f28a9cc0887d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41427 } } }
I20260929 15:56:21.229529 4192 sys_catalog.cc:458] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9 [sys.catalog]: This master's current role is: LEADER
W20260929 15:56:21.233675 4204 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 15:56:21.233798 4204 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 15:56:21.233966 4205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:21.236467 4205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:21.237735 4183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:21.216537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55085 (local address 127.0.0.1:41427)
0929 15:56:21.216751 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:21.216757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:21.217431 (+ 674us) server_negotiation.cc:407] Connection header received
0929 15:56:21.217616 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:21.217621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:21.217704 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:21.217860 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:21.220116 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:21.221316 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:21.223876 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:21.224971 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:21.225125 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:21.229247 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:21.229256 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:21.229268 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:21.229308 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:21.233860 (+ 4552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:21.234884 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:21.234894 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:21.234898 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:21.235101 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:21.235705 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:21.235711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:21.235715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:21.236282 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:21.236590 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:21.237050 (+ 460us) server_negotiation.cc:299] Negotiation successful
0929 15:56:21.237273 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 15:56:21.246420 4205 catalog_manager.cc:1347] Generated new cluster ID: 871f48ae61ee42989fdf67fc7b415627
I20260929 15:56:21.246536 4205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:21.263353 4205 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:21.264685 4205 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:21.276033 4205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c37f24c9e29b431ea96f28a9cc0887d9: Generated new TSK 0
I20260929 15:56:21.277118 4205 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:21.291005 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 4112
I20260929 15:56:21.292408 4175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:21.385650 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 4112
W20260929 15:56:21.403164 4097 connection.cc:535] client connection to 127.0.0.1:41427 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41427): Connection reset by peer (error 104)
2026-09-29T15:56:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2007 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:21 dist-test-slave-c51v krb5kdc[4212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:21 dist-test-slave-c51v krb5kdc[4212](info): set up 2 sockets
Sep 29 15:56:21 dist-test-slave-c51v krb5kdc[4212](info): commencing operation
krb5kdc: starting...
W20260929 15:56:22.505632 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:22 dist-test-slave-c51v krb5kdc[4212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 15:56:22.758876 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44995
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:23.090063 4225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:23.090413 4225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:23.090606 4225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:23.100459 4225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:23.100607 4225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:23.100705 4225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:23.100797 4225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:23.100908 4225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:23.110953 4225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44995
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:23.112380 4225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:23.113669 4225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:23.125304 4231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:23.125386 4230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:23.125386 4233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:23.126813 4225 server_base.cc:1034] running on GCE node
I20260929 15:56:23.127593 4225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:23.128959 4225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:23.130162 4225 hybrid_clock.cc:648] HybridClock initialized: now 1790697383130125 us; error 25 us; skew 500 ppm
I20260929 15:56:23.130470 4225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:23 dist-test-slave-c51v krb5kdc[4212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697383, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:23.133534 4225 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 15:56:23.135816 4225 webserver.cc:460] Webserver started at http://127.0.0.1:32877/ using document root <none> and password file <none>
I20260929 15:56:23.136497 4225 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:23.136624 4225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:23.136950 4225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:23.140021 4225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "666e62b096fc40cba1a42261c8d588b1"
format_stamp: "Formatted at 2026-09-29 15:56:23 on dist-test-slave-c51v"
server_key: "6c4b9b04835197d6829a9629a846ba16"
server_key_iv: "36c9384dbddf4b06e55b639d544a4720"
server_key_version: "encryptionkey@0"
I20260929 15:56:23.140795 4225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "666e62b096fc40cba1a42261c8d588b1"
format_stamp: "Formatted at 2026-09-29 15:56:23 on dist-test-slave-c51v"
server_key: "6c4b9b04835197d6829a9629a846ba16"
server_key_iv: "36c9384dbddf4b06e55b639d544a4720"
server_key_version: "encryptionkey@0"
I20260929 15:56:23.145910 4225 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:56:23.149524 4240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:23.150879 4225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 15:56:23.151078 4225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "666e62b096fc40cba1a42261c8d588b1"
format_stamp: "Formatted at 2026-09-29 15:56:23 on dist-test-slave-c51v"
server_key: "6c4b9b04835197d6829a9629a846ba16"
server_key_iv: "36c9384dbddf4b06e55b639d544a4720"
server_key_version: "encryptionkey@0"
I20260929 15:56:23.151283 4225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:23.188611 4225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:23.193269 4225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:23.193567 4225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:23.211529 4225 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44995
I20260929 15:56:23.211552 4291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44995 every 8 connection(s)
I20260929 15:56:23.212956 4225 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:23.213826 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 4225
I20260929 15:56:23.214031 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:23.214447 16147 external_mini_cluster.cc:1428] Setting key 4661b12ea97bbdfca8b0bc03826c903c
I20260929 15:56:23.217458 4292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:23.229763 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:23.216352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39981 (local address 127.0.0.1:44995)
0929 15:56:23.217115 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:23.217130 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:23.217179 (+ 49us) server_negotiation.cc:407] Connection header received
0929 15:56:23.219562 (+ 2383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:23.219598 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:23.220104 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:23.220574 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:23.221161 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:23.223212 (+ 2051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:23.224533 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:23.225561 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:23.225730 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:23.226888 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:23.226901 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:23.226922 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:23.227686 (+ 764us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:23.227785 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:23.227817 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:23.227995 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:23.228208 (+ 213us) server_negotiation.cc:299] Negotiation successful
0929 15:56:23.228389 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":299,"threads_started":1}
I20260929 15:56:23.231472 4292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1: Bootstrap starting.
I20260929 15:56:23.233642 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:23.234951 4292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:23.236266 4292 log.cc:824] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:23.239348 4292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1: No bootstrap required, opened a new log
I20260929 15:56:23.245728 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:23.239043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52853 (local address 127.0.0.1:44995)
0929 15:56:23.239226 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:23.239232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:23.239678 (+ 446us) server_negotiation.cc:407] Connection header received
0929 15:56:23.239822 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:23.239827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:23.239919 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:23.240037 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:23.240485 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:23.241682 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:23.242983 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:23.243717 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:23.243841 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:23.244937 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:23.244944 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:23.244949 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:23.244970 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:23.245028 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:23.245052 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:23.245167 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:23.245318 (+ 151us) server_negotiation.cc:299] Negotiation successful
0929 15:56:23.245384 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 15:56:23.248980 4292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666e62b096fc40cba1a42261c8d588b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44995 } }
I20260929 15:56:23.249351 4292 raft_consensus.cc:374] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:23.249475 4292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 666e62b096fc40cba1a42261c8d588b1, State: Initialized, Role: FOLLOWER
I20260929 15:56:23.250049 4292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666e62b096fc40cba1a42261c8d588b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44995 } }
I20260929 15:56:23.250272 4292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:23.250427 4292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:23.250609 4292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:23.253149 4292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666e62b096fc40cba1a42261c8d588b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44995 } }
I20260929 15:56:23.253655 4292 leader_election.cc:302] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 666e62b096fc40cba1a42261c8d588b1; no voters:
I20260929 15:56:23.254210 4292 leader_election.cc:288] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:23.254384 4302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:23.255192 4302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [term 1 LEADER]: Becoming Leader. State: Replica: 666e62b096fc40cba1a42261c8d588b1, State: Running, Role: LEADER
I20260929 15:56:23.255777 4302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666e62b096fc40cba1a42261c8d588b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44995 } }
I20260929 15:56:23.256500 4292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:23.259864 4304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 666e62b096fc40cba1a42261c8d588b1. Latest consensus state: current_term: 1 leader_uuid: "666e62b096fc40cba1a42261c8d588b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666e62b096fc40cba1a42261c8d588b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44995 } } }
I20260929 15:56:23.260387 4304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:23.265857 4303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "666e62b096fc40cba1a42261c8d588b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "666e62b096fc40cba1a42261c8d588b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44995 } } }
I20260929 15:56:23.266160 4303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:23.268888 4307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:23.275535 4307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:23.286759 4307 catalog_manager.cc:1347] Generated new cluster ID: 160caddaafbd46ba986605d06b3b1042
I20260929 15:56:23.286949 4307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:23.313582 4307 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:23.315532 4307 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:23.330422 4307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 666e62b096fc40cba1a42261c8d588b1: Generated new TSK 0
I20260929 15:56:23.331553 4307 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:23.340049 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 4225
W20260929 15:56:23.341380 4287 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39981
W20260929 15:56:23.342181 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 4225: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:56:23.342375 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 4225
W20260929 15:56:23.359740 4210 connection.cc:535] client connection to 127.0.0.1:44995 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44995): Connection reset by peer (error 104)
2026-09-29T15:56:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1956 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:23 dist-test-slave-c51v krb5kdc[4323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:23 dist-test-slave-c51v krb5kdc[4323](info): set up 2 sockets
Sep 29 15:56:23 dist-test-slave-c51v krb5kdc[4323](info): commencing operation
krb5kdc: starting...
W20260929 15:56:24.497773 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:24 dist-test-slave-c51v krb5kdc[4323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 15:56:24.744652 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45443
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:25.078864 4336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:25.079144 4336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:25.079350 4336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:25.086455 4336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:25.086571 4336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:25.086691 4336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:25.086795 4336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:25.086886 4336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:25.097107 4336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45443
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:25.098528 4336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:25.099901 4336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:25.111761 4344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:25.111905 4341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:25.111747 4342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:25.112962 4336 server_base.cc:1034] running on GCE node
I20260929 15:56:25.113746 4336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:25.115087 4336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:25.116279 4336 hybrid_clock.cc:648] HybridClock initialized: now 1790697385116219 us; error 56 us; skew 500 ppm
I20260929 15:56:25.116643 4336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:25 dist-test-slave-c51v krb5kdc[4323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697385, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:25.119777 4336 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 15:56:25.121223 4336 webserver.cc:460] Webserver started at http://127.0.0.1:43385/ using document root <none> and password file <none>
I20260929 15:56:25.121901 4336 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:25.122026 4336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:25.122357 4336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:25.125356 4336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "dc52a126e2d8460ab873a008ec1d71d6"
format_stamp: "Formatted at 2026-09-29 15:56:25 on dist-test-slave-c51v"
server_key: "30ae8805b0ba05c8bab9e65531c7abf8"
server_key_iv: "85c78f0a087f7979c9f1cc3b50aad374"
server_key_version: "encryptionkey@0"
I20260929 15:56:25.126137 4336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "dc52a126e2d8460ab873a008ec1d71d6"
format_stamp: "Formatted at 2026-09-29 15:56:25 on dist-test-slave-c51v"
server_key: "30ae8805b0ba05c8bab9e65531c7abf8"
server_key_iv: "85c78f0a087f7979c9f1cc3b50aad374"
server_key_version: "encryptionkey@0"
I20260929 15:56:25.131430 4336 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260929 15:56:25.134732 4351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:25.136119 4336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:25.136305 4336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "dc52a126e2d8460ab873a008ec1d71d6"
format_stamp: "Formatted at 2026-09-29 15:56:25 on dist-test-slave-c51v"
server_key: "30ae8805b0ba05c8bab9e65531c7abf8"
server_key_iv: "85c78f0a087f7979c9f1cc3b50aad374"
server_key_version: "encryptionkey@0"
I20260929 15:56:25.136480 4336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:25.142259 4336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:25.146248 4336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:25.146549 4336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:25.164683 4336 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45443
I20260929 15:56:25.164722 4402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45443 every 8 connection(s)
I20260929 15:56:25.166085 4336 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:25.168691 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 4336
I20260929 15:56:25.168883 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:25.169433 16147 external_mini_cluster.cc:1428] Setting key 1a84a22f9a902fe29093cc7f1bed81d2
I20260929 15:56:25.170408 4403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:25.177022 4406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:25.171149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43631 (local address 127.0.0.1:45443)
0929 15:56:25.171985 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:25.171996 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:25.172036 (+ 40us) server_negotiation.cc:407] Connection header received
0929 15:56:25.173597 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:25.173619 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:25.173954 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:25.174284 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:25.175478 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:25.175484 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:25.175497 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:25.175511 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:25.175584 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:25.175605 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:25.175778 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:25.175882 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 15:56:25.176083 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":418,"threads_started":1}
I20260929 15:56:25.180832 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:25.183629 4403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6: Bootstrap starting.
I20260929 15:56:25.186980 4403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:25.187861 4406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:25.185264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59565 (local address 127.0.0.1:45443)
0929 15:56:25.185480 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:25.185490 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:25.185995 (+ 505us) server_negotiation.cc:407] Connection header received
0929 15:56:25.186119 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:25.186126 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:25.186193 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:25.186311 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:25.187225 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:25.187231 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:25.187237 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:25.187255 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:25.187332 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:25.187359 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:25.187501 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:25.187554 (+ 53us) server_negotiation.cc:299] Negotiation successful
0929 15:56:25.187608 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 15:56:25.188382 4403 log.cc:824] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:25.191470 4403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6: No bootstrap required, opened a new log
I20260929 15:56:25.198612 4403 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc52a126e2d8460ab873a008ec1d71d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45443 } }
I20260929 15:56:25.199008 4403 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:25.199124 4403 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc52a126e2d8460ab873a008ec1d71d6, State: Initialized, Role: FOLLOWER
I20260929 15:56:25.199718 4403 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc52a126e2d8460ab873a008ec1d71d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45443 } }
I20260929 15:56:25.199957 4403 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:25.200117 4403 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:25.200301 4403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:25.202474 4403 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc52a126e2d8460ab873a008ec1d71d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45443 } }
I20260929 15:56:25.202981 4403 leader_election.cc:302] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc52a126e2d8460ab873a008ec1d71d6; no voters:
I20260929 15:56:25.203634 4403 leader_election.cc:288] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:25.203768 4413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:25.205256 4413 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [term 1 LEADER]: Becoming Leader. State: Replica: dc52a126e2d8460ab873a008ec1d71d6, State: Running, Role: LEADER
I20260929 15:56:25.205849 4403 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:25.205756 4413 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc52a126e2d8460ab873a008ec1d71d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45443 } }
I20260929 15:56:25.208568 4414 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc52a126e2d8460ab873a008ec1d71d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc52a126e2d8460ab873a008ec1d71d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45443 } } }
I20260929 15:56:25.208511 4415 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc52a126e2d8460ab873a008ec1d71d6. Latest consensus state: current_term: 1 leader_uuid: "dc52a126e2d8460ab873a008ec1d71d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc52a126e2d8460ab873a008ec1d71d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45443 } } }
I20260929 15:56:25.208993 4414 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:25.209074 4415 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:25.212103 4423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:25.216230 4423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:25.224355 4423 catalog_manager.cc:1347] Generated new cluster ID: 36823e1c88104d8389a4c35677ce45f3
I20260929 15:56:25.224475 4423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:25.246376 4423 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:25.246834 4423 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:25.258507 4423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc52a126e2d8460ab873a008ec1d71d6: Generated new TSK 0
I20260929 15:56:25.259408 4423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:25.266695 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 4336
W20260929 15:56:25.267835 4398 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43631
W20260929 15:56:25.268528 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 4336: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:56:25.268666 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 4336
2026-09-29T15:56:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1926 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:25 dist-test-slave-c51v krb5kdc[4435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:25 dist-test-slave-c51v krb5kdc[4435](info): set up 2 sockets
Sep 29 15:56:25 dist-test-slave-c51v krb5kdc[4435](info): commencing operation
krb5kdc: starting...
W20260929 15:56:26.425994 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:26 dist-test-slave-c51v krb5kdc[4435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 15:56:26.678351 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:27.008054 4448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:27.008417 4448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:27.008590 4448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:27.015882 4448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:27.016043 4448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:27.016148 4448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:27.016253 4448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:27.016330 4448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:27.026141 4448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:27.027570 4448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:27.028936 4448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:27.040911 4453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:27.041033 4457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:27.041157 4455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:27.042078 4448 server_base.cc:1034] running on GCE node
I20260929 15:56:27.042862 4448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:27.044344 4448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:27.045579 4448 hybrid_clock.cc:648] HybridClock initialized: now 1790697387045545 us; error 52 us; skew 500 ppm
I20260929 15:56:27.045943 4448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:27 dist-test-slave-c51v krb5kdc[4435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697387, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:27.052569 4448 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 15:56:27.054055 4448 webserver.cc:460] Webserver started at http://127.0.0.1:32791/ using document root <none> and password file <none>
I20260929 15:56:27.054682 4448 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:27.054781 4448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:27.055035 4448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:27.058091 4448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "4cc4a22fccb847a7afcd18267ad1436e"
format_stamp: "Formatted at 2026-09-29 15:56:27 on dist-test-slave-c51v"
server_key: "2c18714062e318635c217fc8bfa389c2"
server_key_iv: "7b72e37e640288804524eb60d7ed0ae9"
server_key_version: "encryptionkey@0"
I20260929 15:56:27.058804 4448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "4cc4a22fccb847a7afcd18267ad1436e"
format_stamp: "Formatted at 2026-09-29 15:56:27 on dist-test-slave-c51v"
server_key: "2c18714062e318635c217fc8bfa389c2"
server_key_iv: "7b72e37e640288804524eb60d7ed0ae9"
server_key_version: "encryptionkey@0"
I20260929 15:56:27.063776 4448 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260929 15:56:27.066952 4464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:27.068251 4448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:27.068383 4448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "4cc4a22fccb847a7afcd18267ad1436e"
format_stamp: "Formatted at 2026-09-29 15:56:27 on dist-test-slave-c51v"
server_key: "2c18714062e318635c217fc8bfa389c2"
server_key_iv: "7b72e37e640288804524eb60d7ed0ae9"
server_key_version: "encryptionkey@0"
I20260929 15:56:27.068544 4448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:27.080400 4448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:27.084357 4448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:27.084583 4448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:27.102397 4448 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33245
I20260929 15:56:27.102399 4515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33245 every 8 connection(s)
I20260929 15:56:27.103910 4448 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:27.107945 4516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:27.111943 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 4448
I20260929 15:56:27.112138 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:27.112572 16147 external_mini_cluster.cc:1428] Setting key 06325b6a48c93249760b55e29589a3e8
I20260929 15:56:27.118863 4516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e: Bootstrap starting.
I20260929 15:56:27.121922 4516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:27.122902 4516 log.cc:824] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:27.126348 4516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e: No bootstrap required, opened a new log
Sep 29 15:56:27 dist-test-slave-c51v krb5kdc[4435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 15:56:27.137255 4519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:27.114421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44379 (local address 127.0.0.1:33245)
0929 15:56:27.115191 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:27.115204 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:27.115248 (+ 44us) server_negotiation.cc:407] Connection header received
0929 15:56:27.117560 (+ 2312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:27.117584 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:27.117978 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:27.118352 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:27.119986 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:27.121304 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:27.122614 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:27.123475 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:27.123695 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:27.128324 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:27.128337 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:27.128357 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:27.128408 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:27.132500 (+ 4092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:27.133227 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:27.133241 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:27.133258 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:27.133483 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:27.133843 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:27.133847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:27.133851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:27.134618 (+ 767us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:27.134915 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:27.135336 (+ 421us) server_negotiation.cc:299] Negotiation successful
0929 15:56:27.135835 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":356,"threads_started":1}
I20260929 15:56:27.137249 4516 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc4a22fccb847a7afcd18267ad1436e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33245 } }
I20260929 15:56:27.137734 4516 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:27.137859 4516 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cc4a22fccb847a7afcd18267ad1436e, State: Initialized, Role: FOLLOWER
I20260929 15:56:27.138506 4516 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc4a22fccb847a7afcd18267ad1436e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33245 } }
I20260929 15:56:27.138828 4516 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:27.138971 4516 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:27.139160 4516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:27.141314 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:27.142052 4516 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc4a22fccb847a7afcd18267ad1436e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33245 } }
I20260929 15:56:27.142546 4516 leader_election.cc:302] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cc4a22fccb847a7afcd18267ad1436e; no voters:
I20260929 15:56:27.143155 4516 leader_election.cc:288] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:27.143388 4521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:27.144573 4521 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [term 1 LEADER]: Becoming Leader. State: Replica: 4cc4a22fccb847a7afcd18267ad1436e, State: Running, Role: LEADER
I20260929 15:56:27.145560 4516 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:27.145395 4521 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc4a22fccb847a7afcd18267ad1436e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33245 } }
I20260929 15:56:27.154882 4522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cc4a22fccb847a7afcd18267ad1436e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc4a22fccb847a7afcd18267ad1436e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33245 } } }
I20260929 15:56:27.154857 4523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cc4a22fccb847a7afcd18267ad1436e. Latest consensus state: current_term: 1 leader_uuid: "4cc4a22fccb847a7afcd18267ad1436e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cc4a22fccb847a7afcd18267ad1436e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33245 } } }
I20260929 15:56:27.155339 4522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:27.155376 4523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:27.156853 4534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:27.162647 4534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:27.169559 4519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:27.147906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35859 (local address 127.0.0.1:33245)
0929 15:56:27.148064 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:27.148069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:27.148216 (+ 147us) server_negotiation.cc:407] Connection header received
0929 15:56:27.148482 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:27.148491 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:27.148607 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:27.148765 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:27.151351 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:27.152825 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:27.157740 (+ 4915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:27.158610 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:27.158745 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:27.162124 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:27.162131 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:27.162135 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:27.162174 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:27.165997 (+ 3823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:27.166742 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:27.166752 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:27.166756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:27.166887 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:27.167313 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:27.167321 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:27.167327 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:27.167726 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:27.167961 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:27.168749 (+ 788us) server_negotiation.cc:299] Negotiation successful
0929 15:56:27.169035 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 15:56:27.181596 4534 catalog_manager.cc:1347] Generated new cluster ID: f8d954116f794af28f4a3d63d5aaa209
I20260929 15:56:27.181720 4534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:27.189986 4534 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:27.191829 4534 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:27.206027 4534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cc4a22fccb847a7afcd18267ad1436e: Generated new TSK 0
I20260929 15:56:27.206941 4534 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:27.219148 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 4448
I20260929 15:56:27.220423 4511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:27.314239 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 4448
2026-09-29T15:56:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2045 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:27 dist-test-slave-c51v krb5kdc[4549](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:27 dist-test-slave-c51v krb5kdc[4549](info): set up 2 sockets
Sep 29 15:56:27 dist-test-slave-c51v krb5kdc[4549](info): commencing operation
krb5kdc: starting...
W20260929 15:56:28.433681 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.003s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:28 dist-test-slave-c51v krb5kdc[4549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 15:56:28.686684 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42641
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:29.026654 4562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:29.027056 4562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:29.027256 4562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:29.034847 4562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:29.035025 4562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:29.035156 4562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:29.035275 4562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:29.035391 4562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:29.045910 4562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:29.047430 4562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:29.048877 4562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:29.062005 4567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:29.061981 4568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:29.062140 4570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:29.062721 4562 server_base.cc:1034] running on GCE node
I20260929 15:56:29.063633 4562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:29.065061 4562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:29.066293 4562 hybrid_clock.cc:648] HybridClock initialized: now 1790697389066254 us; error 53 us; skew 500 ppm
I20260929 15:56:29.066645 4562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:29 dist-test-slave-c51v krb5kdc[4549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697389, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:29.069823 4562 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 15:56:29.071240 4562 webserver.cc:460] Webserver started at http://127.0.0.1:33189/ using document root <none> and password file <none>
I20260929 15:56:29.071942 4562 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:29.072043 4562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:29.072290 4562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:29.075184 4562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "093ec4165ea441eb9cd14a25dbe17ee5"
format_stamp: "Formatted at 2026-09-29 15:56:29 on dist-test-slave-c51v"
server_key: "a379f8a7054c43ffee82e6c34773924a"
server_key_iv: "13d53cf9ac1a676211f1f1700c2af9e6"
server_key_version: "encryptionkey@0"
I20260929 15:56:29.075973 4562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "093ec4165ea441eb9cd14a25dbe17ee5"
format_stamp: "Formatted at 2026-09-29 15:56:29 on dist-test-slave-c51v"
server_key: "a379f8a7054c43ffee82e6c34773924a"
server_key_iv: "13d53cf9ac1a676211f1f1700c2af9e6"
server_key_version: "encryptionkey@0"
I20260929 15:56:29.081357 4562 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 15:56:29.084735 4577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:29.086033 4562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 15:56:29.086208 4562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "093ec4165ea441eb9cd14a25dbe17ee5"
format_stamp: "Formatted at 2026-09-29 15:56:29 on dist-test-slave-c51v"
server_key: "a379f8a7054c43ffee82e6c34773924a"
server_key_iv: "13d53cf9ac1a676211f1f1700c2af9e6"
server_key_version: "encryptionkey@0"
I20260929 15:56:29.086414 4562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:29.101120 4562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:29.105648 4562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:29.105952 4562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:29.124571 4562 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42641
I20260929 15:56:29.124603 4629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42641 every 8 connection(s)
I20260929 15:56:29.126070 4562 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:29.129988 4630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:29.131670 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 4562
I20260929 15:56:29.131868 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:29.132441 16147 external_mini_cluster.cc:1428] Setting key 8953d28d2f6669d5c4a8cce96d59b860
I20260929 15:56:29.144321 4630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5: Bootstrap starting.
I20260929 15:56:29.145923 4633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:29.134259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53567 (local address 127.0.0.1:42641)
0929 15:56:29.135126 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:29.135138 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:29.135179 (+ 41us) server_negotiation.cc:407] Connection header received
0929 15:56:29.136846 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:29.136880 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:29.137290 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:29.137633 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:29.138192 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:29.139563 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:29.141559 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:29.142264 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:29.142401 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:29.143628 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:29.143635 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:29.143649 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:29.144188 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:29.144267 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:29.144292 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:29.144381 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:29.144642 (+ 261us) server_negotiation.cc:299] Negotiation successful
0929 15:56:29.144951 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":417,"threads_started":1}
I20260929 15:56:29.147718 4630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:29.148911 4630 log.cc:824] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:29.149776 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:29.151893 4630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5: No bootstrap required, opened a new log
I20260929 15:56:29.161034 4633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:29.154385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49585 (local address 127.0.0.1:42641)
0929 15:56:29.154609 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:29.154619 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:29.154962 (+ 343us) server_negotiation.cc:407] Connection header received
0929 15:56:29.155141 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:29.155144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:29.155207 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:29.155340 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:29.155938 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:29.156663 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:29.158562 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:29.159005 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:29.159100 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:29.160319 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:29.160324 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:29.160327 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:29.160355 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:29.160395 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:29.160410 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:29.160480 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:29.160636 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 15:56:29.160705 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260929 15:56:29.161589 4630 raft_consensus.cc:348] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "093ec4165ea441eb9cd14a25dbe17ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } }
I20260929 15:56:29.162155 4630 raft_consensus.cc:374] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:29.162269 4630 raft_consensus.cc:729] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093ec4165ea441eb9cd14a25dbe17ee5, State: Initialized, Role: FOLLOWER
I20260929 15:56:29.162963 4630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "093ec4165ea441eb9cd14a25dbe17ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } }
I20260929 15:56:29.163182 4630 raft_consensus.cc:388] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:29.163334 4630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:29.163480 4630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:29.165678 4630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "093ec4165ea441eb9cd14a25dbe17ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } }
I20260929 15:56:29.166247 4630 leader_election.cc:302] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 093ec4165ea441eb9cd14a25dbe17ee5; no voters:
I20260929 15:56:29.167016 4630 leader_election.cc:288] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:29.167132 4640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:29.167932 4640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [term 1 LEADER]: Becoming Leader. State: Replica: 093ec4165ea441eb9cd14a25dbe17ee5, State: Running, Role: LEADER
I20260929 15:56:29.168529 4640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "093ec4165ea441eb9cd14a25dbe17ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } }
I20260929 15:56:29.169266 4630 sys_catalog.cc:564] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:29.171559 4641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "093ec4165ea441eb9cd14a25dbe17ee5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "093ec4165ea441eb9cd14a25dbe17ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } } }
I20260929 15:56:29.171787 4642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 093ec4165ea441eb9cd14a25dbe17ee5. Latest consensus state: current_term: 1 leader_uuid: "093ec4165ea441eb9cd14a25dbe17ee5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "093ec4165ea441eb9cd14a25dbe17ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42641 } } }
I20260929 15:56:29.172056 4641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:29.172112 4642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:29.174995 4648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:29.180282 4648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:29.189815 4648 catalog_manager.cc:1347] Generated new cluster ID: f38bd1aee6e0401baff3c92a2008b871
I20260929 15:56:29.189962 4648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:29.202445 4648 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:29.203985 4648 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:29.219378 4648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 093ec4165ea441eb9cd14a25dbe17ee5: Generated new TSK 0
I20260929 15:56:29.220554 4648 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:29.238510 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 4562
W20260929 15:56:29.239800 4625 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53567
W20260929 15:56:29.240530 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 4562: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:56:29.240687 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 4562
2026-09-29T15:56:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1928 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:29 dist-test-slave-c51v krb5kdc[4661](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:29 dist-test-slave-c51v krb5kdc[4661](info): set up 2 sockets
Sep 29 15:56:29 dist-test-slave-c51v krb5kdc[4661](info): commencing operation
krb5kdc: starting...
W20260929 15:56:30.341774 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:30 dist-test-slave-c51v krb5kdc[4661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 15:56:30.598908 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33173
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:30.929586 4674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:30.929886 4674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:30.929979 4674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:30.937153 4674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:30.937264 4674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:30.937318 4674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:30.937376 4674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:30.937420 4674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:30.949862 4674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:30.951081 4674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:30.952414 4674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:30.964057 4679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:30.964098 4680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:30.964071 4682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:30.965226 4674 server_base.cc:1034] running on GCE node
I20260929 15:56:30.966059 4674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:30.967533 4674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:30.968731 4674 hybrid_clock.cc:648] HybridClock initialized: now 1790697390968691 us; error 28 us; skew 500 ppm
I20260929 15:56:30.969074 4674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:30 dist-test-slave-c51v krb5kdc[4661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697390, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:30.972076 4674 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 15:56:30.973619 4674 webserver.cc:460] Webserver started at http://127.0.0.1:38603/ using document root <none> and password file <none>
I20260929 15:56:30.974298 4674 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:30.974447 4674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:30.974753 4674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:30.977756 4674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "c86ea8a614e34f78ab322bd189c2e885"
format_stamp: "Formatted at 2026-09-29 15:56:30 on dist-test-slave-c51v"
server_key: "342e9bf44f0fdb754236a015b8d90a23"
server_key_iv: "ae258f267544dca548556e243874be47"
server_key_version: "encryptionkey@0"
I20260929 15:56:30.978545 4674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "c86ea8a614e34f78ab322bd189c2e885"
format_stamp: "Formatted at 2026-09-29 15:56:30 on dist-test-slave-c51v"
server_key: "342e9bf44f0fdb754236a015b8d90a23"
server_key_iv: "ae258f267544dca548556e243874be47"
server_key_version: "encryptionkey@0"
I20260929 15:56:30.983714 4674 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260929 15:56:30.987154 4689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:30.988467 4674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:30.988656 4674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "c86ea8a614e34f78ab322bd189c2e885"
format_stamp: "Formatted at 2026-09-29 15:56:30 on dist-test-slave-c51v"
server_key: "342e9bf44f0fdb754236a015b8d90a23"
server_key_iv: "ae258f267544dca548556e243874be47"
server_key_version: "encryptionkey@0"
I20260929 15:56:30.988852 4674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:30.994534 4674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:30.998363 4674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:30.998667 4674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:31.017299 4674 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33173
I20260929 15:56:31.017395 4740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33173 every 8 connection(s)
I20260929 15:56:31.018735 4674 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:31.022800 4741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:31.024645 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 4674
I20260929 15:56:31.024822 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:31.025280 16147 external_mini_cluster.cc:1428] Setting key 1e04b1de6525f15f681c8a3f92f32009
I20260929 15:56:31.034143 4744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:31.027067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59753 (local address 127.0.0.1:33173)
0929 15:56:31.028044 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:31.028067 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:31.028130 (+ 63us) server_negotiation.cc:407] Connection header received
0929 15:56:31.030257 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:31.030286 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:31.030741 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:31.031213 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:31.032246 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:31.032254 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:31.032271 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:31.032287 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:31.032355 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:31.032380 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:31.032600 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:31.032730 (+ 130us) server_negotiation.cc:299] Negotiation successful
0929 15:56:31.032936 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":405,"threads_started":1}
I20260929 15:56:31.037878 4741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885: Bootstrap starting.
I20260929 15:56:31.037948 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:31.040978 4741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:31.042037 4741 log.cc:824] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:31.045197 4744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:31.042435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51759 (local address 127.0.0.1:33173)
0929 15:56:31.043073 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:31.043080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:31.043207 (+ 127us) server_negotiation.cc:407] Connection header received
0929 15:56:31.043449 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:31.043458 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:31.043518 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:31.043621 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:31.044556 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:31.044562 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:31.044566 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:31.044587 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:31.044633 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:31.044648 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:31.044761 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:31.044885 (+ 124us) server_negotiation.cc:299] Negotiation successful
0929 15:56:31.044942 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520}
I20260929 15:56:31.045549 4741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885: No bootstrap required, opened a new log
I20260929 15:56:31.053131 4741 raft_consensus.cc:348] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86ea8a614e34f78ab322bd189c2e885" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33173 } }
I20260929 15:56:31.053514 4741 raft_consensus.cc:374] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:31.053622 4741 raft_consensus.cc:729] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c86ea8a614e34f78ab322bd189c2e885, State: Initialized, Role: FOLLOWER
I20260929 15:56:31.054214 4741 consensus_queue.cc:260] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86ea8a614e34f78ab322bd189c2e885" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33173 } }
I20260929 15:56:31.054466 4741 raft_consensus.cc:388] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:31.054610 4741 raft_consensus.cc:482] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:31.054800 4741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:31.057132 4741 raft_consensus.cc:504] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86ea8a614e34f78ab322bd189c2e885" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33173 } }
I20260929 15:56:31.057710 4741 leader_election.cc:302] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c86ea8a614e34f78ab322bd189c2e885; no voters:
I20260929 15:56:31.058326 4741 leader_election.cc:288] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:31.058481 4751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:31.059130 4751 raft_consensus.cc:686] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [term 1 LEADER]: Becoming Leader. State: Replica: c86ea8a614e34f78ab322bd189c2e885, State: Running, Role: LEADER
I20260929 15:56:31.059656 4751 consensus_queue.cc:237] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86ea8a614e34f78ab322bd189c2e885" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33173 } }
I20260929 15:56:31.060395 4741 sys_catalog.cc:564] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:31.062479 4754 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c86ea8a614e34f78ab322bd189c2e885. Latest consensus state: current_term: 1 leader_uuid: "c86ea8a614e34f78ab322bd189c2e885" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86ea8a614e34f78ab322bd189c2e885" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33173 } } }
I20260929 15:56:31.062949 4754 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:31.063370 4752 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c86ea8a614e34f78ab322bd189c2e885" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86ea8a614e34f78ab322bd189c2e885" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33173 } } }
I20260929 15:56:31.063684 4752 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:31.065593 4759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:31.070773 4759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:31.079339 4759 catalog_manager.cc:1347] Generated new cluster ID: e7c6102c35434ef2a52a7531384b14b5
I20260929 15:56:31.079494 4759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:31.096140 4759 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:31.096596 4759 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:31.118351 4759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c86ea8a614e34f78ab322bd189c2e885: Generated new TSK 0
I20260929 15:56:31.119524 4759 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:31.127017 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 4674
W20260929 15:56:31.128340 4736 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59753
W20260929 15:56:31.129092 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 4674: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:56:31.129251 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 4674
2026-09-29T15:56:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1887 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:31 dist-test-slave-c51v krb5kdc[4773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:31 dist-test-slave-c51v krb5kdc[4773](info): set up 2 sockets
Sep 29 15:56:31 dist-test-slave-c51v krb5kdc[4773](info): commencing operation
krb5kdc: starting...
W20260929 15:56:32.237665 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:32 dist-test-slave-c51v krb5kdc[4773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 15:56:32.490537 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:32.816499 4786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:32.816784 4786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:32.816968 4786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:32.824065 4786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:32.824187 4786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:32.824294 4786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:32.824400 4786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:32.824492 4786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:32.834239 4786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:32.835594 4786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:32.836930 4786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:32.848954 4791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:32.849007 4794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:32.848912 4792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:32.849699 4786 server_base.cc:1034] running on GCE node
I20260929 15:56:32.850538 4786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:32.852016 4786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:32.853308 4786 hybrid_clock.cc:648] HybridClock initialized: now 1790697392853205 us; error 56 us; skew 500 ppm
I20260929 15:56:32.853678 4786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:32 dist-test-slave-c51v krb5kdc[4773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697392, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:32.860167 4786 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 15:56:32.861606 4786 webserver.cc:460] Webserver started at http://127.0.0.1:46775/ using document root <none> and password file <none>
I20260929 15:56:32.862272 4786 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:32.862386 4786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:32.862654 4786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:32.865768 4786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "4f7de3529c75456389056928bd546b11"
format_stamp: "Formatted at 2026-09-29 15:56:32 on dist-test-slave-c51v"
server_key: "5a96846e3f4f8f8f3d89574d595f597f"
server_key_iv: "8ba327be26738bf03aadd13b678f18be"
server_key_version: "encryptionkey@0"
I20260929 15:56:32.866493 4786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "4f7de3529c75456389056928bd546b11"
format_stamp: "Formatted at 2026-09-29 15:56:32 on dist-test-slave-c51v"
server_key: "5a96846e3f4f8f8f3d89574d595f597f"
server_key_iv: "8ba327be26738bf03aadd13b678f18be"
server_key_version: "encryptionkey@0"
I20260929 15:56:32.871968 4786 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 15:56:32.875389 4801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:32.876710 4786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:56:32.876848 4786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "4f7de3529c75456389056928bd546b11"
format_stamp: "Formatted at 2026-09-29 15:56:32 on dist-test-slave-c51v"
server_key: "5a96846e3f4f8f8f3d89574d595f597f"
server_key_iv: "8ba327be26738bf03aadd13b678f18be"
server_key_version: "encryptionkey@0"
I20260929 15:56:32.877002 4786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:32.891774 4786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:32.896142 4786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:32.896395 4786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:32.914454 4786 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36675
I20260929 15:56:32.914443 4852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36675 every 8 connection(s)
I20260929 15:56:32.915992 4786 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:32.919970 4853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:32.924744 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 4786
I20260929 15:56:32.924942 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:32.925383 16147 external_mini_cluster.cc:1428] Setting key 70bcae441565a5a517a37d6773757355
I20260929 15:56:32.930827 4853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11: Bootstrap starting.
I20260929 15:56:32.934474 4853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:32.935729 4853 log.cc:824] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:32.939249 4853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11: No bootstrap required, opened a new log
Sep 29 15:56:32 dist-test-slave-c51v krb5kdc[4773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 15:56:32.951153 4853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7de3529c75456389056928bd546b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36675 } }
I20260929 15:56:32.951637 4853 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:32.951768 4853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f7de3529c75456389056928bd546b11, State: Initialized, Role: FOLLOWER
I20260929 15:56:32.952409 4853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7de3529c75456389056928bd546b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36675 } }
I20260929 15:56:32.952661 4853 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:32.952811 4853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:32.953008 4853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:32.953184 4856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:32.927098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.65:56739 (local address 127.0.0.1:36675)
0929 15:56:32.927932 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:32.927943 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:32.927990 (+ 47us) server_negotiation.cc:407] Connection header received
0929 15:56:32.929893 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:32.929918 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:32.930313 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:32.930637 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:32.932507 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:32.933866 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:32.935659 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:32.936678 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:32.936903 (+ 225us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:32.942375 (+ 5472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:32.942400 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:32.942419 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:32.942464 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:32.947643 (+ 5179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:32.948785 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:32.948795 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:32.948807 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:32.949062 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:32.949700 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:32.949705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:32.949709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:32.950437 (+ 728us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:32.950770 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:32.951250 (+ 480us) server_negotiation.cc:299] Negotiation successful
0929 15:56:32.951676 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":390,"threads_started":1}
I20260929 15:56:32.956073 4853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7de3529c75456389056928bd546b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36675 } }
I20260929 15:56:32.956601 4853 leader_election.cc:302] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f7de3529c75456389056928bd546b11; no voters:
I20260929 15:56:32.957242 4853 leader_election.cc:288] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:32.957451 4858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:32.957842 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:32.959198 4853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:32.959580 4858 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [term 1 LEADER]: Becoming Leader. State: Replica: 4f7de3529c75456389056928bd546b11, State: Running, Role: LEADER
I20260929 15:56:32.960189 4858 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7de3529c75456389056928bd546b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36675 } }
I20260929 15:56:32.965305 4860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f7de3529c75456389056928bd546b11. Latest consensus state: current_term: 1 leader_uuid: "4f7de3529c75456389056928bd546b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7de3529c75456389056928bd546b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36675 } } }
I20260929 15:56:32.965687 4860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:32.966338 4859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f7de3529c75456389056928bd546b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7de3529c75456389056928bd546b11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36675 } } }
I20260929 15:56:32.966713 4859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:32.969470 4871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:32.978650 4871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:32.993419 4856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:32.965213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.65:57261 (local address 127.0.0.1:36675)
0929 15:56:32.965429 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:32.965441 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:32.972194 (+ 6753us) server_negotiation.cc:407] Connection header received
0929 15:56:32.972352 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:32.972362 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:32.972478 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:32.972617 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:32.975005 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:32.975952 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:32.977975 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:32.979066 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:32.979205 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:32.983813 (+ 4608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:32.983820 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:32.983824 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:32.983860 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:32.988111 (+ 4251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:32.989375 (+ 1264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:32.989381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:32.989385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:32.989557 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:32.990283 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:32.990291 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:32.990297 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:32.991020 (+ 723us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:32.991420 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:32.992650 (+ 1230us) server_negotiation.cc:299] Negotiation successful
0929 15:56:32.992893 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 15:56:33.001026 4871 catalog_manager.cc:1347] Generated new cluster ID: 669ef4cd1b24460c8b2c4c6732ebd7da
I20260929 15:56:33.001233 4871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:33.022218 4871 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:33.023658 4871 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:33.039886 4871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f7de3529c75456389056928bd546b11: Generated new TSK 0
I20260929 15:56:33.040724 4871 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:33.050626 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 4786
I20260929 15:56:33.052248 4848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:33.148298 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 4786
W20260929 15:56:33.163268 4771 connection.cc:535] client connection to 127.0.0.1:36675 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36675): Connection reset by peer (error 104)
2026-09-29T15:56:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2016 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:33 dist-test-slave-c51v krb5kdc[4886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:33 dist-test-slave-c51v krb5kdc[4886](info): set up 2 sockets
Sep 29 15:56:33 dist-test-slave-c51v krb5kdc[4886](info): commencing operation
krb5kdc: starting...
W20260929 15:56:34.249650 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:34 dist-test-slave-c51v krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 15:56:34.510448 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:34.837733 4899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:34.838002 4899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:34.838121 4899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:34.845214 4899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:34.845343 4899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:34.845446 4899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:34.845541 4899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:34.845623 4899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:34.855197 4899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:34.856549 4899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:34.857861 4899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:34.869774 4906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:34.869858 4908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:34.869805 4905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:34.870607 4899 server_base.cc:1034] running on GCE node
I20260929 15:56:34.871407 4899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:34.872867 4899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:34.874059 4899 hybrid_clock.cc:648] HybridClock initialized: now 1790697394874002 us; error 54 us; skew 500 ppm
I20260929 15:56:34.874360 4899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:34 dist-test-slave-c51v krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697394, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:34.877545 4899 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 15:56:34.878959 4899 webserver.cc:460] Webserver started at http://127.0.0.1:46687/ using document root <none> and password file <none>
I20260929 15:56:34.879711 4899 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:34.879848 4899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:34.880199 4899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:34.883193 4899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "0bb468b4832847b28ae73a34884f7f6e"
format_stamp: "Formatted at 2026-09-29 15:56:34 on dist-test-slave-c51v"
server_key: "b41fa394b33e97dbfd5f134d0af9f7ec"
server_key_iv: "c415ede7107fb9adae0d68c20adcfc7a"
server_key_version: "encryptionkey@0"
I20260929 15:56:34.884127 4899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "0bb468b4832847b28ae73a34884f7f6e"
format_stamp: "Formatted at 2026-09-29 15:56:34 on dist-test-slave-c51v"
server_key: "b41fa394b33e97dbfd5f134d0af9f7ec"
server_key_iv: "c415ede7107fb9adae0d68c20adcfc7a"
server_key_version: "encryptionkey@0"
I20260929 15:56:34.889287 4899 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:56:34.892668 4915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:34.893996 4899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260929 15:56:34.894186 4899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "0bb468b4832847b28ae73a34884f7f6e"
format_stamp: "Formatted at 2026-09-29 15:56:34 on dist-test-slave-c51v"
server_key: "b41fa394b33e97dbfd5f134d0af9f7ec"
server_key_iv: "c415ede7107fb9adae0d68c20adcfc7a"
server_key_version: "encryptionkey@0"
I20260929 15:56:34.894390 4899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:34.907649 4899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:34.911513 4899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:34.911789 4899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:34.930171 4899 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36323
I20260929 15:56:34.930202 4966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36323 every 8 connection(s)
I20260929 15:56:34.931595 4899 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:34.935109 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 4899
I20260929 15:56:34.935329 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:34.935590 4967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:34.935942 16147 external_mini_cluster.cc:1428] Setting key 9e3589be9914bdf1d775396720d3ddc6
I20260929 15:56:34.951424 4967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e: Bootstrap starting.
I20260929 15:56:34.951786 4970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:34.937697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.65:58351 (local address 127.0.0.1:36323)
0929 15:56:34.938519 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:34.938531 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:34.938573 (+ 42us) server_negotiation.cc:407] Connection header received
0929 15:56:34.940313 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:34.940341 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:34.940870 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:34.941307 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:34.942149 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:34.944188 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:34.946271 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:34.947744 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:34.947946 (+ 202us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:34.949075 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:34.949088 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:34.949108 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:34.949819 (+ 711us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:34.949912 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:34.949951 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:34.950098 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:34.950305 (+ 207us) server_negotiation.cc:299] Negotiation successful
0929 15:56:34.950487 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":358,"threads_started":1}
I20260929 15:56:34.955154 4967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:34.955612 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:34.956256 4967 log.cc:824] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:34.959169 4967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e: No bootstrap required, opened a new log
I20260929 15:56:34.967219 4970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:34.960319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.65:58951 (local address 127.0.0.1:36323)
0929 15:56:34.960526 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:34.960534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:34.961087 (+ 553us) server_negotiation.cc:407] Connection header received
0929 15:56:34.961284 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:34.961288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:34.961352 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:34.961437 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:34.962005 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:34.962771 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:34.964694 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:34.965152 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:34.965247 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:34.966454 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:34.966459 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:34.966462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:34.966478 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:34.966512 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:34.966527 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:34.966607 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:34.966815 (+ 208us) server_negotiation.cc:299] Negotiation successful
0929 15:56:34.966858 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 15:56:34.968910 4967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb468b4832847b28ae73a34884f7f6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36323 } }
I20260929 15:56:34.969297 4967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:34.969409 4967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bb468b4832847b28ae73a34884f7f6e, State: Initialized, Role: FOLLOWER
I20260929 15:56:34.970021 4967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb468b4832847b28ae73a34884f7f6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36323 } }
I20260929 15:56:34.970271 4967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:34.970409 4967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:34.970593 4967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:34.972834 4967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb468b4832847b28ae73a34884f7f6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36323 } }
I20260929 15:56:34.973348 4967 leader_election.cc:302] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bb468b4832847b28ae73a34884f7f6e; no voters:
I20260929 15:56:34.973915 4967 leader_election.cc:288] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:34.974114 4977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:34.975613 4977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [term 1 LEADER]: Becoming Leader. State: Replica: 0bb468b4832847b28ae73a34884f7f6e, State: Running, Role: LEADER
I20260929 15:56:34.976162 4967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:34.976130 4977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb468b4832847b28ae73a34884f7f6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36323 } }
I20260929 15:56:34.979557 4978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bb468b4832847b28ae73a34884f7f6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb468b4832847b28ae73a34884f7f6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36323 } } }
I20260929 15:56:34.979537 4979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bb468b4832847b28ae73a34884f7f6e. Latest consensus state: current_term: 1 leader_uuid: "0bb468b4832847b28ae73a34884f7f6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bb468b4832847b28ae73a34884f7f6e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36323 } } }
I20260929 15:56:34.979969 4979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:34.979969 4978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:34.983827 4986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:34.989006 4986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:34.998615 4986 catalog_manager.cc:1347] Generated new cluster ID: d59afbd975a5406eb0260ab603b05651
I20260929 15:56:34.998736 4986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:35.022029 4986 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:35.023422 4986 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:35.029995 4986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bb468b4832847b28ae73a34884f7f6e: Generated new TSK 0
I20260929 15:56:35.030845 4986 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:35.041903 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 4899
W20260929 15:56:35.042972 4962 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.65:58351
W20260929 15:56:35.043776 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 4899: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:56:35.043938 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 4899
W20260929 15:56:35.061044 4884 connection.cc:535] client connection to 127.0.0.1:36323 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36323): Connection reset by peer (error 104)
2026-09-29T15:56:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1897 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:35 dist-test-slave-c51v krb5kdc[4998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:35 dist-test-slave-c51v krb5kdc[4998](info): set up 2 sockets
Sep 29 15:56:35 dist-test-slave-c51v krb5kdc[4998](info): commencing operation
krb5kdc: starting...
W20260929 15:56:36.165849 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:36 dist-test-slave-c51v krb5kdc[4998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 15:56:36.423233 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44493
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:36.748788 5011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:36.749117 5011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:36.749287 5011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:36.756462 5011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:36.756613 5011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:36.756711 5011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:36.756839 5011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:36.756925 5011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:36.766728 5011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:36.768078 5011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:36.769405 5011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:36.781464 5017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:36.781646 5016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:36.782145 5019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:36.782403 5011 server_base.cc:1034] running on GCE node
I20260929 15:56:36.783162 5011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:36.784539 5011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:36.785771 5011 hybrid_clock.cc:648] HybridClock initialized: now 1790697396785738 us; error 46 us; skew 500 ppm
I20260929 15:56:36.786116 5011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:36 dist-test-slave-c51v krb5kdc[4998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697396, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:36.789204 5011 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 15:56:36.790638 5011 webserver.cc:460] Webserver started at http://127.0.0.1:34211/ using document root <none> and password file <none>
I20260929 15:56:36.791414 5011 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:36.791564 5011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:36.791879 5011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:36.794847 5011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "b7f874fc6add4959bbc41c8fbaa003fb"
format_stamp: "Formatted at 2026-09-29 15:56:36 on dist-test-slave-c51v"
server_key: "a81a9124ba81642ed6fefe912d4e4a55"
server_key_iv: "c286972e2c084cf22930a6922b1deffc"
server_key_version: "encryptionkey@0"
I20260929 15:56:36.795696 5011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "b7f874fc6add4959bbc41c8fbaa003fb"
format_stamp: "Formatted at 2026-09-29 15:56:36 on dist-test-slave-c51v"
server_key: "a81a9124ba81642ed6fefe912d4e4a55"
server_key_iv: "c286972e2c084cf22930a6922b1deffc"
server_key_version: "encryptionkey@0"
I20260929 15:56:36.800745 5011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260929 15:56:36.804198 5026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:36.805486 5011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:56:36.805675 5011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "b7f874fc6add4959bbc41c8fbaa003fb"
format_stamp: "Formatted at 2026-09-29 15:56:36 on dist-test-slave-c51v"
server_key: "a81a9124ba81642ed6fefe912d4e4a55"
server_key_iv: "c286972e2c084cf22930a6922b1deffc"
server_key_version: "encryptionkey@0"
I20260929 15:56:36.805881 5011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:36.811614 5011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:36.815639 5011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:36.815935 5011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:36.834084 5011 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44493
I20260929 15:56:36.834127 5078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44493 every 8 connection(s)
I20260929 15:56:36.835605 5011 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:36.837369 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 5011
I20260929 15:56:36.837548 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:36.838018 16147 external_mini_cluster.cc:1428] Setting key 8230bb0e90ab4e04fcd4d4bb0764607f
I20260929 15:56:36.839811 5079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:36.847302 5082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:36.839949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.65:40125 (local address 127.0.0.1:44493)
0929 15:56:36.840774 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:36.840789 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:36.840856 (+ 67us) server_negotiation.cc:407] Connection header received
0929 15:56:36.842953 (+ 2097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:36.842984 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:36.843497 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:36.844053 (+ 556us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:36.845130 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:36.845140 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:36.845158 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:36.845182 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:36.845298 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:36.845325 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:36.845580 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:36.845727 (+ 147us) server_negotiation.cc:299] Negotiation successful
0929 15:56:36.845940 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":325,"threads_started":1}
I20260929 15:56:36.850945 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:36.854318 5079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb: Bootstrap starting.
I20260929 15:56:36.857950 5082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:36.855499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.65:47029 (local address 127.0.0.1:44493)
0929 15:56:36.855682 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:36.855690 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:36.856028 (+ 338us) server_negotiation.cc:407] Connection header received
0929 15:56:36.856151 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:36.856157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:36.856230 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:36.856363 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:36.857269 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:36.857275 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:36.857279 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:36.857301 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:36.857350 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:36.857373 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:36.857495 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:36.857596 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 15:56:36.857654 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 15:56:36.858314 5079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:36.859612 5079 log.cc:824] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:36.862484 5079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb: No bootstrap required, opened a new log
I20260929 15:56:36.871035 5079 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f874fc6add4959bbc41c8fbaa003fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44493 } }
I20260929 15:56:36.871482 5079 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:36.871610 5079 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7f874fc6add4959bbc41c8fbaa003fb, State: Initialized, Role: FOLLOWER
I20260929 15:56:36.872234 5079 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f874fc6add4959bbc41c8fbaa003fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44493 } }
I20260929 15:56:36.872490 5079 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:36.872663 5079 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:36.872864 5079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:36.875268 5079 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f874fc6add4959bbc41c8fbaa003fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44493 } }
I20260929 15:56:36.875874 5079 leader_election.cc:302] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7f874fc6add4959bbc41c8fbaa003fb; no voters:
I20260929 15:56:36.876533 5079 leader_election.cc:288] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:36.876677 5089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:36.878196 5089 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [term 1 LEADER]: Becoming Leader. State: Replica: b7f874fc6add4959bbc41c8fbaa003fb, State: Running, Role: LEADER
I20260929 15:56:36.878741 5089 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f874fc6add4959bbc41c8fbaa003fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44493 } }
I20260929 15:56:36.879004 5079 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:36.881314 5090 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7f874fc6add4959bbc41c8fbaa003fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f874fc6add4959bbc41c8fbaa003fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44493 } } }
I20260929 15:56:36.881669 5090 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:36.882241 5091 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7f874fc6add4959bbc41c8fbaa003fb. Latest consensus state: current_term: 1 leader_uuid: "b7f874fc6add4959bbc41c8fbaa003fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f874fc6add4959bbc41c8fbaa003fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44493 } } }
I20260929 15:56:36.882525 5091 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:36.882664 5096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:36.889298 5096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:36.898660 5096 catalog_manager.cc:1347] Generated new cluster ID: dcbab13ac020480395b63a0225d9bd5d
I20260929 15:56:36.898797 5096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:36.919404 5096 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:36.919836 5096 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:36.926190 5096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7f874fc6add4959bbc41c8fbaa003fb: Generated new TSK 0
I20260929 15:56:36.927008 5096 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:36.938979 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 5011
W20260929 15:56:36.943601 5074 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.65:40125
W20260929 15:56:36.944370 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 5011: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:56:36.944547 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 5011
2026-09-29T15:56:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1901 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:37 dist-test-slave-c51v krb5kdc[5110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:37 dist-test-slave-c51v krb5kdc[5110](info): set up 2 sockets
Sep 29 15:56:37 dist-test-slave-c51v krb5kdc[5110](info): commencing operation
krb5kdc: starting...
W20260929 15:56:38.053784 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:38 dist-test-slave-c51v krb5kdc[5110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 15:56:38.314219 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36607
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:38.647022 5123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:38.647342 5123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:38.647519 5123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:38.654886 5123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:38.655000 5123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:38.655046 5123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:38.655138 5123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:38.655227 5123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:38.665187 5123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36607
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:38.666641 5123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:38.668004 5123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:38.680080 5128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:38.680204 5129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:38.680177 5131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:38.681444 5123 server_base.cc:1034] running on GCE node
I20260929 15:56:38.682281 5123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:38.683800 5123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:38.685041 5123 hybrid_clock.cc:648] HybridClock initialized: now 1790697398684995 us; error 42 us; skew 500 ppm
I20260929 15:56:38.685392 5123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:38 dist-test-slave-c51v krb5kdc[5110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697398, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:38.692065 5123 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 15:56:38.693632 5123 webserver.cc:460] Webserver started at http://127.0.0.1:42421/ using document root <none> and password file <none>
I20260929 15:56:38.694347 5123 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:38.694501 5123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:38.694813 5123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:38.697844 5123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "25bc87a18ee746c2a535237f04ece278"
format_stamp: "Formatted at 2026-09-29 15:56:38 on dist-test-slave-c51v"
server_key: "d494926698ccc9442d38f9f4fb2e8b0c"
server_key_iv: "a5d7216894118f144825b65a6444f3a5"
server_key_version: "encryptionkey@0"
I20260929 15:56:38.698642 5123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "25bc87a18ee746c2a535237f04ece278"
format_stamp: "Formatted at 2026-09-29 15:56:38 on dist-test-slave-c51v"
server_key: "d494926698ccc9442d38f9f4fb2e8b0c"
server_key_iv: "a5d7216894118f144825b65a6444f3a5"
server_key_version: "encryptionkey@0"
I20260929 15:56:38.703783 5123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 15:56:38.707170 5138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:38.708545 5123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:38.708740 5123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "25bc87a18ee746c2a535237f04ece278"
format_stamp: "Formatted at 2026-09-29 15:56:38 on dist-test-slave-c51v"
server_key: "d494926698ccc9442d38f9f4fb2e8b0c"
server_key_iv: "a5d7216894118f144825b65a6444f3a5"
server_key_version: "encryptionkey@0"
I20260929 15:56:38.708941 5123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:38.725340 5123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:38.729540 5123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:38.729823 5123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:38.747999 5123 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36607
I20260929 15:56:38.748023 5189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36607 every 8 connection(s)
I20260929 15:56:38.749388 5123 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:38.753116 5190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:38.757730 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 5123
I20260929 15:56:38.757932 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:38.758522 16147 external_mini_cluster.cc:1428] Setting key febeb84cb2e6e36e0712d3ded104a126
I20260929 15:56:38.765868 5190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278: Bootstrap starting.
I20260929 15:56:38.769650 5190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:38.771019 5190 log.cc:824] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:38.774032 5190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278: No bootstrap required, opened a new log
Sep 29 15:56:38 dist-test-slave-c51v krb5kdc[5110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 15:56:38.783609 5193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:38.760352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.65:37985 (local address 127.0.0.1:36607)
0929 15:56:38.761177 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:38.761193 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:38.761248 (+ 55us) server_negotiation.cc:407] Connection header received
0929 15:56:38.764071 (+ 2823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:38.764101 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:38.764671 (+ 570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:38.765122 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:38.766573 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:38.768670 (+ 2097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:38.769997 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:38.771074 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:38.771249 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:38.775838 (+ 4589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:38.775847 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:38.775862 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:38.775910 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:38.779157 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:38.780129 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:38.780139 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:38.780145 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:38.780309 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:38.780780 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:38.780783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:38.780785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:38.781349 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:38.781618 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:38.782145 (+ 527us) server_negotiation.cc:299] Negotiation successful
0929 15:56:38.782499 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":370,"threads_started":1}
I20260929 15:56:38.783532 5190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bc87a18ee746c2a535237f04ece278" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36607 } }
I20260929 15:56:38.783984 5190 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:38.784119 5190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25bc87a18ee746c2a535237f04ece278, State: Initialized, Role: FOLLOWER
I20260929 15:56:38.784684 5190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bc87a18ee746c2a535237f04ece278" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36607 } }
I20260929 15:56:38.784948 5190 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:38.785111 5190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:38.785324 5190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:38.787961 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:38.788146 5190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bc87a18ee746c2a535237f04ece278" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36607 } }
I20260929 15:56:38.788691 5190 leader_election.cc:302] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25bc87a18ee746c2a535237f04ece278; no voters:
I20260929 15:56:38.789310 5190 leader_election.cc:288] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:38.789449 5195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:38.790656 5195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [term 1 LEADER]: Becoming Leader. State: Replica: 25bc87a18ee746c2a535237f04ece278, State: Running, Role: LEADER
I20260929 15:56:38.791319 5195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bc87a18ee746c2a535237f04ece278" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36607 } }
I20260929 15:56:38.791688 5190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:38.794099 5196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25bc87a18ee746c2a535237f04ece278" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bc87a18ee746c2a535237f04ece278" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36607 } } }
I20260929 15:56:38.794422 5196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:38.793989 5197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25bc87a18ee746c2a535237f04ece278. Latest consensus state: current_term: 1 leader_uuid: "25bc87a18ee746c2a535237f04ece278" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bc87a18ee746c2a535237f04ece278" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36607 } } }
I20260929 15:56:38.795204 5197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:38.797833 5205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:38.807658 5205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:38.819044 5193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:38.800050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.65:44829 (local address 127.0.0.1:36607)
0929 15:56:38.800265 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:38.800274 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:38.800304 (+ 30us) server_negotiation.cc:407] Connection header received
0929 15:56:38.800607 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:38.800617 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:38.800727 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:38.800818 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:38.803634 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:38.804804 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:38.806609 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:38.807569 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:38.807696 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:38.811240 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:38.811246 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:38.811251 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:38.811317 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:38.815342 (+ 4025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:38.816380 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:38.816386 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:38.816390 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:38.816523 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:38.817104 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:38.817109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:38.817113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:38.817559 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:38.817800 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:38.818512 (+ 712us) server_negotiation.cc:299] Negotiation successful
0929 15:56:38.818696 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 15:56:38.823345 5205 catalog_manager.cc:1347] Generated new cluster ID: 2eb70c5ce4ff4b2aaa88b6978f2cfce4
I20260929 15:56:38.823478 5205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:38.841424 5205 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:38.843035 5205 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:38.849393 5205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25bc87a18ee746c2a535237f04ece278: Generated new TSK 0
I20260929 15:56:38.850206 5205 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:38.865662 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 5123
I20260929 15:56:38.867086 5185 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:38.963505 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 5123
W20260929 15:56:38.980882 5108 connection.cc:535] client connection to 127.0.0.1:36607 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36607): Connection reset by peer (error 104)
2026-09-29T15:56:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2018 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:39 dist-test-slave-c51v krb5kdc[5223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:39 dist-test-slave-c51v krb5kdc[5223](info): set up 2 sockets
Sep 29 15:56:39 dist-test-slave-c51v krb5kdc[5223](info): commencing operation
krb5kdc: starting...
W20260929 15:56:40.069842 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:40 dist-test-slave-c51v krb5kdc[5223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 15:56:40.333091 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41101
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:40.661305 5236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:40.661636 5236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:40.661818 5236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:40.668809 5236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:40.668968 5236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:40.669080 5236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:40.669196 5236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:40.669296 5236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:40.678917 5236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41101
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:40.680290 5236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:40.681620 5236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:40.693166 5243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:40.693221 5245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:40.693413 5242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:40.694024 5236 server_base.cc:1034] running on GCE node
I20260929 15:56:40.694802 5236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:40.696247 5236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:40.697499 5236 hybrid_clock.cc:648] HybridClock initialized: now 1790697400697459 us; error 49 us; skew 500 ppm
I20260929 15:56:40.697877 5236 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:40 dist-test-slave-c51v krb5kdc[5223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697400, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:40.701122 5236 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 15:56:40.702615 5236 webserver.cc:460] Webserver started at http://127.0.0.1:37153/ using document root <none> and password file <none>
I20260929 15:56:40.703388 5236 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:40.703533 5236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:40.703855 5236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:40.706866 5236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "0a5059145ade4a0285965ec86140b506"
format_stamp: "Formatted at 2026-09-29 15:56:40 on dist-test-slave-c51v"
server_key: "b02122c4619b12f28a663918fb14121a"
server_key_iv: "0d54c0d76051643740f7e165d8449486"
server_key_version: "encryptionkey@0"
I20260929 15:56:40.707711 5236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "0a5059145ade4a0285965ec86140b506"
format_stamp: "Formatted at 2026-09-29 15:56:40 on dist-test-slave-c51v"
server_key: "b02122c4619b12f28a663918fb14121a"
server_key_iv: "0d54c0d76051643740f7e165d8449486"
server_key_version: "encryptionkey@0"
I20260929 15:56:40.712966 5236 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 15:56:40.716512 5252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:40.717803 5236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 15:56:40.718001 5236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "0a5059145ade4a0285965ec86140b506"
format_stamp: "Formatted at 2026-09-29 15:56:40 on dist-test-slave-c51v"
server_key: "b02122c4619b12f28a663918fb14121a"
server_key_iv: "0d54c0d76051643740f7e165d8449486"
server_key_version: "encryptionkey@0"
I20260929 15:56:40.718197 5236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:40.741744 5236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:40.746140 5236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:40.746430 5236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:40.764814 5236 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41101
I20260929 15:56:40.764846 5303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41101 every 8 connection(s)
I20260929 15:56:40.766250 5236 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:40.768585 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 5236
I20260929 15:56:40.768774 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:40.769297 16147 external_mini_cluster.cc:1428] Setting key 9a0b08ee4bb138d8a04c1332d13e3830
I20260929 15:56:40.770624 5304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:40.782362 5307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:40.771073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.65:39887 (local address 127.0.0.1:41101)
0929 15:56:40.771818 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:40.771829 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:40.771869 (+ 40us) server_negotiation.cc:407] Connection header received
0929 15:56:40.773420 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:40.773442 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:40.773822 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:40.774157 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:40.774741 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:40.776129 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:40.777985 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:40.778642 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:40.778770 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:40.780088 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:40.780095 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:40.780112 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:40.780660 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:40.780738 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:40.780764 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:40.780881 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:40.781210 (+ 329us) server_negotiation.cc:299] Negotiation successful
0929 15:56:40.781409 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":352,"threads_started":1}
I20260929 15:56:40.783800 5304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506: Bootstrap starting.
I20260929 15:56:40.786226 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:40.787145 5304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:40.788156 5304 log.cc:824] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:40.791049 5304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506: No bootstrap required, opened a new log
I20260929 15:56:40.797574 5307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:40.790632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.65:51909 (local address 127.0.0.1:41101)
0929 15:56:40.790810 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:40.790815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:40.791333 (+ 518us) server_negotiation.cc:407] Connection header received
0929 15:56:40.791567 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:40.791575 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:40.791662 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:40.791775 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:40.792295 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:40.793529 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:40.794801 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:40.795546 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:40.795667 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:40.796717 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:40.796725 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:40.796732 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:40.796759 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:40.796820 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:40.796844 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:40.796972 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:40.797150 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 15:56:40.797202 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 15:56:40.800235 5304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5059145ade4a0285965ec86140b506" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41101 } }
I20260929 15:56:40.800632 5304 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:40.800751 5304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a5059145ade4a0285965ec86140b506, State: Initialized, Role: FOLLOWER
I20260929 15:56:40.801369 5304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5059145ade4a0285965ec86140b506" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41101 } }
I20260929 15:56:40.801614 5304 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:40.801749 5304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:40.801936 5304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:40.804168 5304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5059145ade4a0285965ec86140b506" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41101 } }
I20260929 15:56:40.804663 5304 leader_election.cc:302] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a5059145ade4a0285965ec86140b506; no voters:
I20260929 15:56:40.805241 5304 leader_election.cc:288] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:40.805399 5315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:40.806119 5315 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [term 1 LEADER]: Becoming Leader. State: Replica: 0a5059145ade4a0285965ec86140b506, State: Running, Role: LEADER
I20260929 15:56:40.806694 5315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5059145ade4a0285965ec86140b506" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41101 } }
I20260929 15:56:40.807310 5304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:40.809445 5317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a5059145ade4a0285965ec86140b506. Latest consensus state: current_term: 1 leader_uuid: "0a5059145ade4a0285965ec86140b506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5059145ade4a0285965ec86140b506" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41101 } } }
I20260929 15:56:40.809768 5317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:40.810644 5321 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:40.810513 5316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a5059145ade4a0285965ec86140b506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a5059145ade4a0285965ec86140b506" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41101 } } }
I20260929 15:56:40.811746 5316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:40.816478 5321 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:40.825996 5321 catalog_manager.cc:1347] Generated new cluster ID: c5abb22cdbb14a0ab71adc37cf9d4661
I20260929 15:56:40.826110 5321 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:40.843302 5321 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:40.844604 5321 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:40.852306 5321 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a5059145ade4a0285965ec86140b506: Generated new TSK 0
I20260929 15:56:40.853173 5321 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:40.863484 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 5236
W20260929 15:56:40.864807 5299 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.65:39887
W20260929 15:56:40.865612 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 5236: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:56:40.865787 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 5236
W20260929 15:56:40.883512 5221 connection.cc:535] client connection to 127.0.0.1:41101 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41101): Connection reset by peer (error 104)
2026-09-29T15:56:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1902 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:40 dist-test-slave-c51v krb5kdc[5336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:40 dist-test-slave-c51v krb5kdc[5336](info): set up 2 sockets
Sep 29 15:56:40 dist-test-slave-c51v krb5kdc[5336](info): commencing operation
krb5kdc: starting...
W20260929 15:56:41.973839 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.002s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:42 dist-test-slave-c51v krb5kdc[5336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 15:56:42.227324 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40839
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:42.558053 5349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:42.558355 5349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:42.558449 5349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:42.565683 5349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:42.565804 5349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:42.565852 5349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:42.565917 5349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:42.565954 5349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:42.575605 5349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40839
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:42.576913 5349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:42.578207 5349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:42.590328 5357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:42.590454 5355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:42.590440 5354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:42.591388 5349 server_base.cc:1034] running on GCE node
I20260929 15:56:42.592175 5349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:42.593712 5349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:42.594964 5349 hybrid_clock.cc:648] HybridClock initialized: now 1790697402594919 us; error 42 us; skew 500 ppm
I20260929 15:56:42.595388 5349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:42 dist-test-slave-c51v krb5kdc[5336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697402, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:42.598639 5349 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 15:56:42.600191 5349 webserver.cc:460] Webserver started at http://127.0.0.1:42985/ using document root <none> and password file <none>
I20260929 15:56:42.600894 5349 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:42.601038 5349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:42.601358 5349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:42.604485 5349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "e951bae1d9c74302915ab125131c5d53"
format_stamp: "Formatted at 2026-09-29 15:56:42 on dist-test-slave-c51v"
server_key: "e859f1f46bc162b40fb6859f00695b9b"
server_key_iv: "1a85132c046c885baa3b0207c9b6bf8c"
server_key_version: "encryptionkey@0"
I20260929 15:56:42.605255 5349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "e951bae1d9c74302915ab125131c5d53"
format_stamp: "Formatted at 2026-09-29 15:56:42 on dist-test-slave-c51v"
server_key: "e859f1f46bc162b40fb6859f00695b9b"
server_key_iv: "1a85132c046c885baa3b0207c9b6bf8c"
server_key_version: "encryptionkey@0"
I20260929 15:56:42.610486 5349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 15:56:42.614055 5364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:42.615540 5349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:42.615726 5349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "e951bae1d9c74302915ab125131c5d53"
format_stamp: "Formatted at 2026-09-29 15:56:42 on dist-test-slave-c51v"
server_key: "e859f1f46bc162b40fb6859f00695b9b"
server_key_iv: "1a85132c046c885baa3b0207c9b6bf8c"
server_key_version: "encryptionkey@0"
I20260929 15:56:42.615942 5349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:42.621639 5349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:42.625475 5349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:42.625766 5349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:42.643975 5349 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40839
I20260929 15:56:42.644052 5416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40839 every 8 connection(s)
I20260929 15:56:42.645376 5349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:42.648993 5417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:42.649849 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 5349
I20260929 15:56:42.650034 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:42.650457 16147 external_mini_cluster.cc:1428] Setting key c273dbde41eb489e259cafb52a4371b1
I20260929 15:56:42.659215 5420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:42.652315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.65:49177 (local address 127.0.0.1:40839)
0929 15:56:42.653081 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:42.653095 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:42.653150 (+ 55us) server_negotiation.cc:407] Connection header received
0929 15:56:42.655573 (+ 2423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:42.655601 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:42.656045 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:42.656499 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:42.657414 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:42.657422 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:42.657438 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:42.657454 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:42.657512 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:42.657538 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:42.657730 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:42.657857 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 15:56:42.658065 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":280,"threads_started":1}
I20260929 15:56:42.662673 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:42.662911 5417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53: Bootstrap starting.
I20260929 15:56:42.665940 5417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:42.667164 5417 log.cc:824] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:42.669646 5420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:42.667038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.65:33327 (local address 127.0.0.1:40839)
0929 15:56:42.667229 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:42.667236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:42.667632 (+ 396us) server_negotiation.cc:407] Connection header received
0929 15:56:42.667814 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:42.667819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:42.667900 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:42.668012 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:42.668899 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:42.668904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:42.668908 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:42.668922 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:42.668957 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:42.668971 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:42.669070 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:42.669277 (+ 207us) server_negotiation.cc:299] Negotiation successful
0929 15:56:42.669362 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 15:56:42.670397 5417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53: No bootstrap required, opened a new log
I20260929 15:56:42.677771 5417 raft_consensus.cc:348] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e951bae1d9c74302915ab125131c5d53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40839 } }
I20260929 15:56:42.678133 5417 raft_consensus.cc:374] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:42.678246 5417 raft_consensus.cc:729] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e951bae1d9c74302915ab125131c5d53, State: Initialized, Role: FOLLOWER
I20260929 15:56:42.678792 5417 consensus_queue.cc:260] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e951bae1d9c74302915ab125131c5d53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40839 } }
I20260929 15:56:42.679029 5417 raft_consensus.cc:388] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:42.679173 5417 raft_consensus.cc:482] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:42.679368 5417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:42.681491 5417 raft_consensus.cc:504] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e951bae1d9c74302915ab125131c5d53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40839 } }
I20260929 15:56:42.681979 5417 leader_election.cc:302] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e951bae1d9c74302915ab125131c5d53; no voters:
I20260929 15:56:42.682539 5417 leader_election.cc:288] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:42.682717 5427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:42.683884 5427 raft_consensus.cc:686] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [term 1 LEADER]: Becoming Leader. State: Replica: e951bae1d9c74302915ab125131c5d53, State: Running, Role: LEADER
I20260929 15:56:42.684383 5427 consensus_queue.cc:237] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e951bae1d9c74302915ab125131c5d53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40839 } }
I20260929 15:56:42.684765 5417 sys_catalog.cc:564] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:42.687237 5429 sys_catalog.cc:455] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e951bae1d9c74302915ab125131c5d53. Latest consensus state: current_term: 1 leader_uuid: "e951bae1d9c74302915ab125131c5d53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e951bae1d9c74302915ab125131c5d53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40839 } } }
I20260929 15:56:42.687650 5429 sys_catalog.cc:458] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:42.687546 5428 sys_catalog.cc:455] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e951bae1d9c74302915ab125131c5d53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e951bae1d9c74302915ab125131c5d53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40839 } } }
I20260929 15:56:42.687924 5428 sys_catalog.cc:458] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:42.690881 5436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:42.695179 5436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:42.703455 5436 catalog_manager.cc:1347] Generated new cluster ID: 08d2f9be4d184b938fa0ed65d2acd482
I20260929 15:56:42.703595 5436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:42.718338 5436 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:42.718783 5436 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:42.731153 5436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e951bae1d9c74302915ab125131c5d53: Generated new TSK 0
I20260929 15:56:42.731933 5436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:42.742491 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 5349
W20260929 15:56:42.743538 5412 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.65:49177
W20260929 15:56:42.744206 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 5349: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:56:42.744354 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 5349
2026-09-29T15:56:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1877 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:42 dist-test-slave-c51v krb5kdc[5448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:42 dist-test-slave-c51v krb5kdc[5448](info): set up 2 sockets
Sep 29 15:56:42 dist-test-slave-c51v krb5kdc[5448](info): commencing operation
krb5kdc: starting...
W20260929 15:56:43.865710 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.002s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:44 dist-test-slave-c51v krb5kdc[5448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:56:44.126641 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:38333
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:44205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:38333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:44.461143 5461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:44.461494 5461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:44.461654 5461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:44.469414 5461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:44.469581 5461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:44.469691 5461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:44.469796 5461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:44.469868 5461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:44.479880 5461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:44205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:38333
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:38333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:44.481227 5461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:44.482621 5461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:44.494794 5466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:44.494733 5467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:44.495333 5469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:44.495790 5461 server_base.cc:1034] running on GCE node
I20260929 15:56:44.496579 5461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:44.498106 5461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:44.499425 5461 hybrid_clock.cc:648] HybridClock initialized: now 1790697404499380 us; error 50 us; skew 500 ppm
I20260929 15:56:44.499795 5461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:44 dist-test-slave-c51v krb5kdc[5448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697404, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:44.505829 5461 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:56:44.507483 5461 webserver.cc:460] Webserver started at http://127.15.196.254:34855/ using document root <none> and password file <none>
I20260929 15:56:44.508205 5461 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:44.508358 5461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:44.508673 5461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:44.511759 5461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "90c1da6501654662813c0068912c0192"
format_stamp: "Formatted at 2026-09-29 15:56:44 on dist-test-slave-c51v"
server_key: "2dc9819de30ae54c07d2efa0a054b813"
server_key_iv: "24938f04c53a0ef26a1e2afbcd9f33a2"
server_key_version: "encryptionkey@0"
I20260929 15:56:44.512521 5461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "90c1da6501654662813c0068912c0192"
format_stamp: "Formatted at 2026-09-29 15:56:44 on dist-test-slave-c51v"
server_key: "2dc9819de30ae54c07d2efa0a054b813"
server_key_iv: "24938f04c53a0ef26a1e2afbcd9f33a2"
server_key_version: "encryptionkey@0"
I20260929 15:56:44.517460 5461 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260929 15:56:44.520920 5476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:44.522290 5461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260929 15:56:44.522467 5461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "90c1da6501654662813c0068912c0192"
format_stamp: "Formatted at 2026-09-29 15:56:44 on dist-test-slave-c51v"
server_key: "2dc9819de30ae54c07d2efa0a054b813"
server_key_iv: "24938f04c53a0ef26a1e2afbcd9f33a2"
server_key_version: "encryptionkey@0"
I20260929 15:56:44.522651 5461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:44.552124 5461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:44.556110 5461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:44.556402 5461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:44.574820 5461 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:38333
I20260929 15:56:44.574851 5527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:38333 every 8 connection(s)
I20260929 15:56:44.576236 5461 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:44.579967 5528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:44.581683 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 5461
I20260929 15:56:44.581871 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:44.582430 16147 external_mini_cluster.cc:1428] Setting key 07e3abb7c920cf662df8c58a8a7e9239
I20260929 15:56:44.594297 5528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192: Bootstrap starting.
Sep 29 15:56:44 dist-test-slave-c51v krb5kdc[5448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:56:44.598001 5528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:44.599447 5528 log.cc:824] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:44.602734 5528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192: No bootstrap required, opened a new log
I20260929 15:56:44.606429 5531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:44.584231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51101 (local address 127.15.196.254:38333)
0929 15:56:44.585061 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:44.585072 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:44.585115 (+ 43us) server_negotiation.cc:407] Connection header received
0929 15:56:44.586690 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:44.586716 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:44.587126 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:44.587510 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:44.589166 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:44.590430 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:44.592248 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:44.592894 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:44.593016 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:44.597782 (+ 4766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:44.597796 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:44.597818 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:44.597866 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:44.601799 (+ 3933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:44.602543 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:44.602550 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:44.602557 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:44.602773 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:44.603343 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:44.603348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:44.603351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:44.604038 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:44.604359 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:44.604753 (+ 394us) server_negotiation.cc:299] Negotiation successful
0929 15:56:44.605083 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":403,"threads_started":1}
I20260929 15:56:44.610424 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:44.612370 5528 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c1da6501654662813c0068912c0192" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38333 } }
I20260929 15:56:44.612731 5528 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:44.612860 5528 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c1da6501654662813c0068912c0192, State: Initialized, Role: FOLLOWER
I20260929 15:56:44.613572 5528 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c1da6501654662813c0068912c0192" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38333 } }
I20260929 15:56:44.613839 5528 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:44.613977 5528 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:44.614152 5528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:44.616859 5528 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c1da6501654662813c0068912c0192" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38333 } }
I20260929 15:56:44.617457 5528 leader_election.cc:302] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90c1da6501654662813c0068912c0192; no voters:
I20260929 15:56:44.618149 5528 leader_election.cc:288] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:44.618283 5539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:44.619752 5539 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [term 1 LEADER]: Becoming Leader. State: Replica: 90c1da6501654662813c0068912c0192, State: Running, Role: LEADER
I20260929 15:56:44.620332 5539 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c1da6501654662813c0068912c0192" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38333 } }
I20260929 15:56:44.620479 5528 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:44.624872 5540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90c1da6501654662813c0068912c0192" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c1da6501654662813c0068912c0192" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38333 } } }
I20260929 15:56:44.625283 5540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:44.625828 5541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90c1da6501654662813c0068912c0192. Latest consensus state: current_term: 1 leader_uuid: "90c1da6501654662813c0068912c0192" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c1da6501654662813c0068912c0192" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38333 } } }
I20260929 15:56:44.626230 5541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:44.627942 5548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:44.634371 5548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:44.634402 5531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:44.614929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41487 (local address 127.15.196.254:38333)
0929 15:56:44.615136 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:44.615142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:44.615537 (+ 395us) server_negotiation.cc:407] Connection header received
0929 15:56:44.615666 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:44.615671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:44.615749 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:44.615855 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:44.617316 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:44.618626 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:44.620486 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:44.621161 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:44.621291 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:44.625941 (+ 4650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:44.625953 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:44.625962 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:44.626012 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:44.630779 (+ 4767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:44.631858 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:44.631864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:44.631867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:44.632069 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:44.632670 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:44.632676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:44.632679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:44.633140 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:44.633439 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:44.633827 (+ 388us) server_negotiation.cc:299] Negotiation successful
0929 15:56:44.634011 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 15:56:44.643783 5548 catalog_manager.cc:1347] Generated new cluster ID: a29cd9a7026e48d8ba5725a67c251ad8
I20260929 15:56:44.643921 5548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:44.657234 5548 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:44.659071 5548 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:44.667519 5548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90c1da6501654662813c0068912c0192: Generated new TSK 0
I20260929 15:56:44.668315 5548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:44.679716 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 5461
I20260929 15:56:44.681372 5523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:44.772120 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 5461
W20260929 15:56:44.789631 5446 connection.cc:535] client connection to 127.15.196.254:38333 recv error: Network error: failed to read from TLS socket (remote: 127.15.196.254:38333): Connection reset by peer (error 104)
2026-09-29T15:56:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2028 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:44 dist-test-slave-c51v krb5kdc[5561](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:44 dist-test-slave-c51v krb5kdc[5561](info): set up 2 sockets
Sep 29 15:56:44 dist-test-slave-c51v krb5kdc[5561](info): commencing operation
krb5kdc: starting...
W20260929 15:56:45.885653 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.002s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:46 dist-test-slave-c51v krb5kdc[5561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:56:46.140872 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:33717
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:36533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:33717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:46.469475 5574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:46.469926 5574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:46.470093 5574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:46.477406 5574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:46.477558 5574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:46.477663 5574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:46.477773 5574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:46.477842 5574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:46.488152 5574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:36533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:33717
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:33717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:46.489467 5574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:46.490777 5574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:46.502674 5580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:46.502765 5582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:46.502792 5579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:46.503507 5574 server_base.cc:1034] running on GCE node
I20260929 15:56:46.504309 5574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:46.505792 5574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:46.507014 5574 hybrid_clock.cc:648] HybridClock initialized: now 1790697406506974 us; error 32 us; skew 500 ppm
I20260929 15:56:46.507411 5574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:46 dist-test-slave-c51v krb5kdc[5561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697406, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:46.510473 5574 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:56:46.511929 5574 webserver.cc:460] Webserver started at http://127.15.196.254:34441/ using document root <none> and password file <none>
I20260929 15:56:46.512619 5574 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:46.512763 5574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:46.513068 5574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:46.516124 5574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "96244fe3cd0a4d5d8e9f3548b35edafe"
format_stamp: "Formatted at 2026-09-29 15:56:46 on dist-test-slave-c51v"
server_key: "126d94204fda0d22823d5c463ad2590b"
server_key_iv: "6783635caadb7aefdefc4d09fc4ff011"
server_key_version: "encryptionkey@0"
I20260929 15:56:46.516892 5574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "96244fe3cd0a4d5d8e9f3548b35edafe"
format_stamp: "Formatted at 2026-09-29 15:56:46 on dist-test-slave-c51v"
server_key: "126d94204fda0d22823d5c463ad2590b"
server_key_iv: "6783635caadb7aefdefc4d09fc4ff011"
server_key_version: "encryptionkey@0"
I20260929 15:56:46.521800 5574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 15:56:46.525070 5589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:46.526316 5574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 15:56:46.526499 5574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "96244fe3cd0a4d5d8e9f3548b35edafe"
format_stamp: "Formatted at 2026-09-29 15:56:46 on dist-test-slave-c51v"
server_key: "126d94204fda0d22823d5c463ad2590b"
server_key_iv: "6783635caadb7aefdefc4d09fc4ff011"
server_key_version: "encryptionkey@0"
I20260929 15:56:46.526690 5574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:46.546101 5574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:46.550536 5574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:46.550823 5574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:46.569458 5574 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:33717
I20260929 15:56:46.569478 5640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:33717 every 8 connection(s)
I20260929 15:56:46.570883 5574 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:46.574702 5641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:46.574767 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 5574
I20260929 15:56:46.574959 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:46.575479 16147 external_mini_cluster.cc:1428] Setting key 3847be0a65f02708a817766c10f87321
I20260929 15:56:46.590037 5641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe: Bootstrap starting.
I20260929 15:56:46.590086 5644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:46.577310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57239 (local address 127.15.196.254:33717)
0929 15:56:46.578152 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:46.578171 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:46.578219 (+ 48us) server_negotiation.cc:407] Connection header received
0929 15:56:46.580122 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:46.580153 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:46.580743 (+ 590us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:46.581180 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:46.581713 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:46.583760 (+ 2047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:46.585041 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:46.586079 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:46.586291 (+ 212us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:46.587444 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:46.587454 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:46.587471 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:46.588185 (+ 714us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:46.588271 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:46.588306 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:46.588462 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:46.588681 (+ 219us) server_negotiation.cc:299] Negotiation successful
0929 15:56:46.588845 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":350,"threads_started":1}
I20260929 15:56:46.593341 5641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:46.593837 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:46.594491 5641 log.cc:824] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:46.597121 5641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe: No bootstrap required, opened a new log
I20260929 15:56:46.605084 5644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:46.598209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44143 (local address 127.15.196.254:33717)
0929 15:56:46.598410 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:46.598416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:46.598886 (+ 470us) server_negotiation.cc:407] Connection header received
0929 15:56:46.599086 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:46.599092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:46.599152 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:46.599225 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:46.599902 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:46.600661 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:46.602516 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:46.602978 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:46.603070 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:46.604313 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:46.604318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:46.604321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:46.604338 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:46.604388 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:46.604403 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:46.604483 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:46.604682 (+ 199us) server_negotiation.cc:299] Negotiation successful
0929 15:56:46.604726 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 15:56:46.607035 5641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96244fe3cd0a4d5d8e9f3548b35edafe" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 33717 } }
I20260929 15:56:46.607496 5641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:46.607610 5641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96244fe3cd0a4d5d8e9f3548b35edafe, State: Initialized, Role: FOLLOWER
I20260929 15:56:46.608237 5641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96244fe3cd0a4d5d8e9f3548b35edafe" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 33717 } }
I20260929 15:56:46.608485 5641 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:46.608635 5641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:46.608815 5641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:46.611241 5641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96244fe3cd0a4d5d8e9f3548b35edafe" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 33717 } }
I20260929 15:56:46.611810 5641 leader_election.cc:302] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96244fe3cd0a4d5d8e9f3548b35edafe; no voters:
I20260929 15:56:46.612375 5641 leader_election.cc:288] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:46.612532 5651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:46.613204 5651 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [term 1 LEADER]: Becoming Leader. State: Replica: 96244fe3cd0a4d5d8e9f3548b35edafe, State: Running, Role: LEADER
I20260929 15:56:46.613746 5651 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96244fe3cd0a4d5d8e9f3548b35edafe" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 33717 } }
I20260929 15:56:46.614610 5641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:46.616786 5653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96244fe3cd0a4d5d8e9f3548b35edafe. Latest consensus state: current_term: 1 leader_uuid: "96244fe3cd0a4d5d8e9f3548b35edafe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96244fe3cd0a4d5d8e9f3548b35edafe" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 33717 } } }
I20260929 15:56:46.617002 5652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96244fe3cd0a4d5d8e9f3548b35edafe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96244fe3cd0a4d5d8e9f3548b35edafe" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 33717 } } }
I20260929 15:56:46.617238 5653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:46.617280 5652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:46.618705 5659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:46.624333 5659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:46.633036 5659 catalog_manager.cc:1347] Generated new cluster ID: ec0fe3fe826d4c21ba841d44462ac973
I20260929 15:56:46.633138 5659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:46.645010 5659 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:46.646809 5659 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:46.657469 5659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96244fe3cd0a4d5d8e9f3548b35edafe: Generated new TSK 0
I20260929 15:56:46.658504 5659 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:46.669988 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 5574
W20260929 15:56:46.671095 5636 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57239
W20260929 15:56:46.672078 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 5574: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:56:46.672227 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 5574
W20260929 15:56:46.686612 5559 connection.cc:535] client connection to 127.15.196.254:33717 recv error: Network error: failed to read from TLS socket (remote: 127.15.196.254:33717): Connection reset by peer (error 104)
2026-09-29T15:56:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1896 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:46 dist-test-slave-c51v krb5kdc[5672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:46 dist-test-slave-c51v krb5kdc[5672](info): set up 2 sockets
Sep 29 15:56:46 dist-test-slave-c51v krb5kdc[5672](info): commencing operation
krb5kdc: starting...
W20260929 15:56:47.765766 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:47 dist-test-slave-c51v krb5kdc[5672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:56:48.022706 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:34305
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:34305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:48.361189 5686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:48.361552 5686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:48.361724 5686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:48.369326 5686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:48.369483 5686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:48.369594 5686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:48.369705 5686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:48.369788 5686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:48.380213 5686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:34305
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:34305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:48.381639 5686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:48.383023 5686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:48.394971 5691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:48.394989 5694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:48.394956 5692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:48.396777 5686 server_base.cc:1034] running on GCE node
I20260929 15:56:48.397542 5686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:48.399001 5686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:48.400231 5686 hybrid_clock.cc:648] HybridClock initialized: now 1790697408400168 us; error 89 us; skew 500 ppm
I20260929 15:56:48.400583 5686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:48 dist-test-slave-c51v krb5kdc[5672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697408, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:48.403947 5686 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:56:48.405455 5686 webserver.cc:460] Webserver started at http://127.15.196.254:40011/ using document root <none> and password file <none>
I20260929 15:56:48.406183 5686 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:48.406327 5686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:48.406628 5686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:48.409746 5686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "7df639d784e14fb5af7fa8be01ae93f2"
format_stamp: "Formatted at 2026-09-29 15:56:48 on dist-test-slave-c51v"
server_key: "8af6cf02ed2ac08e25317b69325e9afe"
server_key_iv: "e32c9a5831a643b7e7be74ed1bb011d4"
server_key_version: "encryptionkey@0"
I20260929 15:56:48.410516 5686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "7df639d784e14fb5af7fa8be01ae93f2"
format_stamp: "Formatted at 2026-09-29 15:56:48 on dist-test-slave-c51v"
server_key: "8af6cf02ed2ac08e25317b69325e9afe"
server_key_iv: "e32c9a5831a643b7e7be74ed1bb011d4"
server_key_version: "encryptionkey@0"
I20260929 15:56:48.415625 5686 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 15:56:48.419019 5701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:48.420406 5686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:48.420599 5686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "7df639d784e14fb5af7fa8be01ae93f2"
format_stamp: "Formatted at 2026-09-29 15:56:48 on dist-test-slave-c51v"
server_key: "8af6cf02ed2ac08e25317b69325e9afe"
server_key_iv: "e32c9a5831a643b7e7be74ed1bb011d4"
server_key_version: "encryptionkey@0"
I20260929 15:56:48.420805 5686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:48.426457 5686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:48.430784 5686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:48.431089 5686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:48.449671 5686 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:34305
I20260929 15:56:48.449695 5752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:34305 every 8 connection(s)
I20260929 15:56:48.451143 5686 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:48.454938 5753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:48.457111 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 5686
I20260929 15:56:48.457290 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:48.457731 16147 external_mini_cluster.cc:1428] Setting key a0dce528c700eaa40f1b51431874b0d4
I20260929 15:56:48.465459 5756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:48.459544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34167 (local address 127.15.196.254:34305)
0929 15:56:48.460350 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:48.460361 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:48.460407 (+ 46us) server_negotiation.cc:407] Connection header received
0929 15:56:48.462060 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:48.462082 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:48.462411 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:48.462739 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:48.463877 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:48.463884 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:48.463897 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:48.463918 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:48.463972 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:48.463993 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:48.464154 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:48.464310 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 15:56:48.464549 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":375,"threads_started":1}
I20260929 15:56:48.467214 5753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2: Bootstrap starting.
I20260929 15:56:48.469133 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:48.470538 5753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:48.471587 5753 log.cc:824] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:48.474766 5753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2: No bootstrap required, opened a new log
I20260929 15:56:48.475872 5756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:48.473440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52329 (local address 127.15.196.254:34305)
0929 15:56:48.473650 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:48.473655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:48.473980 (+ 325us) server_negotiation.cc:407] Connection header received
0929 15:56:48.474141 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:48.474144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:48.474187 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:48.474260 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:48.475208 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:48.475213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:48.475216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:48.475229 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:48.475266 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:48.475283 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:48.475421 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:48.475537 (+ 116us) server_negotiation.cc:299] Negotiation successful
0929 15:56:48.475578 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 15:56:48.482352 5753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df639d784e14fb5af7fa8be01ae93f2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34305 } }
I20260929 15:56:48.482760 5753 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:48.482868 5753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df639d784e14fb5af7fa8be01ae93f2, State: Initialized, Role: FOLLOWER
I20260929 15:56:48.483513 5753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df639d784e14fb5af7fa8be01ae93f2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34305 } }
I20260929 15:56:48.483748 5753 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:48.483893 5753 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:48.484079 5753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:48.486196 5753 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df639d784e14fb5af7fa8be01ae93f2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34305 } }
I20260929 15:56:48.486696 5753 leader_election.cc:302] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7df639d784e14fb5af7fa8be01ae93f2; no voters:
I20260929 15:56:48.487432 5753 leader_election.cc:288] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:48.487534 5763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:48.488301 5763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [term 1 LEADER]: Becoming Leader. State: Replica: 7df639d784e14fb5af7fa8be01ae93f2, State: Running, Role: LEADER
I20260929 15:56:48.488909 5763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df639d784e14fb5af7fa8be01ae93f2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34305 } }
I20260929 15:56:48.489848 5753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:48.492163 5764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7df639d784e14fb5af7fa8be01ae93f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df639d784e14fb5af7fa8be01ae93f2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34305 } } }
I20260929 15:56:48.492678 5764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:48.493249 5765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7df639d784e14fb5af7fa8be01ae93f2. Latest consensus state: current_term: 1 leader_uuid: "7df639d784e14fb5af7fa8be01ae93f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df639d784e14fb5af7fa8be01ae93f2" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 34305 } } }
I20260929 15:56:48.493696 5765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:48.494204 5770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:48.502942 5770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:48.512102 5770 catalog_manager.cc:1347] Generated new cluster ID: 805f14c013044e758ad310550dd31c56
I20260929 15:56:48.512234 5770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:48.528498 5770 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:48.528970 5770 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:48.540843 5770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7df639d784e14fb5af7fa8be01ae93f2: Generated new TSK 0
I20260929 15:56:48.541731 5770 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:48.552871 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 5686
W20260929 15:56:48.554073 5748 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34167
W20260929 15:56:48.554747 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 5686: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:56:48.554898 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 5686
2026-09-29T15:56:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1885 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:48 dist-test-slave-c51v krb5kdc[5784](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:48 dist-test-slave-c51v krb5kdc[5784](info): set up 2 sockets
Sep 29 15:56:48 dist-test-slave-c51v krb5kdc[5784](info): commencing operation
krb5kdc: starting...
W20260929 15:56:49.665786 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.005s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:49 dist-test-slave-c51v krb5kdc[5784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:56:49.920584 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:37423
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:41017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:37423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:50.249593 5797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:50.249929 5797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:50.250190 5797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:50.258177 5797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:50.258352 5797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:50.258457 5797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:50.258589 5797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:50.258683 5797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:50.269480 5797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:41017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:37423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:37423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:50.270844 5797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:50.272248 5797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:50.284286 5802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:50.284246 5803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:50.284309 5805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:50.285003 5797 server_base.cc:1034] running on GCE node
I20260929 15:56:50.285869 5797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:50.287436 5797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:50.288717 5797 hybrid_clock.cc:648] HybridClock initialized: now 1790697410288670 us; error 40 us; skew 500 ppm
I20260929 15:56:50.289140 5797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:50 dist-test-slave-c51v krb5kdc[5784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697410, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:50.297580 5797 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:56:50.299226 5797 webserver.cc:460] Webserver started at http://127.15.196.254:41745/ using document root <none> and password file <none>
I20260929 15:56:50.300001 5797 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:50.300149 5797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:50.300469 5797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:50.303676 5797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "0852f6f706bd4356bb978956365bfa96"
format_stamp: "Formatted at 2026-09-29 15:56:50 on dist-test-slave-c51v"
server_key: "0ff761de12fd688e4de2637f0af1e624"
server_key_iv: "2fc9f0281de38eb3b4f4055fac7f8b71"
server_key_version: "encryptionkey@0"
I20260929 15:56:50.304458 5797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "0852f6f706bd4356bb978956365bfa96"
format_stamp: "Formatted at 2026-09-29 15:56:50 on dist-test-slave-c51v"
server_key: "0ff761de12fd688e4de2637f0af1e624"
server_key_iv: "2fc9f0281de38eb3b4f4055fac7f8b71"
server_key_version: "encryptionkey@0"
I20260929 15:56:50.309576 5797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 15:56:50.312906 5812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:50.314236 5797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 15:56:50.314419 5797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "0852f6f706bd4356bb978956365bfa96"
format_stamp: "Formatted at 2026-09-29 15:56:50 on dist-test-slave-c51v"
server_key: "0ff761de12fd688e4de2637f0af1e624"
server_key_iv: "2fc9f0281de38eb3b4f4055fac7f8b71"
server_key_version: "encryptionkey@0"
I20260929 15:56:50.314692 5797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:50.338673 5797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:50.342761 5797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:50.343072 5797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:50.361732 5797 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:37423
I20260929 15:56:50.361756 5864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:37423 every 8 connection(s)
I20260929 15:56:50.363140 5797 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:50.366014 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 5797
I20260929 15:56:50.366231 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:50.366833 16147 external_mini_cluster.cc:1428] Setting key 25dd4bf438d742a467c8495520dbcc0e
I20260929 15:56:50.367585 5865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 15:56:50 dist-test-slave-c51v krb5kdc[5784](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:56:50.384173 5865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96: Bootstrap starting.
I20260929 15:56:50.388108 5865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:50.389303 5865 log.cc:824] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:50.391494 5868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:50.368696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56597 (local address 127.15.196.254:37423)
0929 15:56:50.369483 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:50.369494 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:50.369534 (+ 40us) server_negotiation.cc:407] Connection header received
0929 15:56:50.371088 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:50.371122 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:50.371643 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:50.371981 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:50.373942 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:50.375950 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:50.377890 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:50.378569 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:50.378703 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:50.383986 (+ 5283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:50.383995 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:50.384011 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:50.384053 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:50.387162 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:50.387931 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:50.387938 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:50.387945 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:50.388120 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:50.388509 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:50.388515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:50.388518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:50.389058 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:50.389362 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:50.389822 (+ 460us) server_negotiation.cc:299] Negotiation successful
0929 15:56:50.390429 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":372,"threads_started":1}
I20260929 15:56:50.392611 5865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96: No bootstrap required, opened a new log
I20260929 15:56:50.395195 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:50.402058 5865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0852f6f706bd4356bb978956365bfa96" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 37423 } }
I20260929 15:56:50.402623 5865 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:50.402782 5865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0852f6f706bd4356bb978956365bfa96, State: Initialized, Role: FOLLOWER
I20260929 15:56:50.403541 5865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0852f6f706bd4356bb978956365bfa96" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 37423 } }
I20260929 15:56:50.403836 5865 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:50.404011 5865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:50.404238 5865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:50.407732 5865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0852f6f706bd4356bb978956365bfa96" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 37423 } }
I20260929 15:56:50.408437 5865 leader_election.cc:302] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0852f6f706bd4356bb978956365bfa96; no voters:
I20260929 15:56:50.409191 5865 leader_election.cc:288] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:50.409265 5875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:50.411149 5875 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [term 1 LEADER]: Becoming Leader. State: Replica: 0852f6f706bd4356bb978956365bfa96, State: Running, Role: LEADER
I20260929 15:56:50.411824 5875 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0852f6f706bd4356bb978956365bfa96" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 37423 } }
I20260929 15:56:50.411988 5865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:50.414654 5877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0852f6f706bd4356bb978956365bfa96. Latest consensus state: current_term: 1 leader_uuid: "0852f6f706bd4356bb978956365bfa96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0852f6f706bd4356bb978956365bfa96" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 37423 } } }
I20260929 15:56:50.415055 5877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:50.415612 5876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0852f6f706bd4356bb978956365bfa96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0852f6f706bd4356bb978956365bfa96" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 37423 } } }
I20260929 15:56:50.416697 5876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:50.420292 5880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:50.420570 5868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:50.399701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50091 (local address 127.15.196.254:37423)
0929 15:56:50.399926 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:50.399933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:50.400529 (+ 596us) server_negotiation.cc:407] Connection header received
0929 15:56:50.400725 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:50.400729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:50.400795 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:50.400876 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:50.403007 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:50.403964 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:50.405951 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:50.406401 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:50.406505 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:50.409964 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:50.409971 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:50.409976 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:50.410025 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:50.414353 (+ 4328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:50.415948 (+ 1595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:50.415959 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:50.415965 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:50.416744 (+ 779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:50.417479 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:50.417487 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:50.417494 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:50.417997 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:50.418307 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:50.418785 (+ 478us) server_negotiation.cc:299] Negotiation successful
0929 15:56:50.419076 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 15:56:50.433079 5880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:50.443238 5880 catalog_manager.cc:1347] Generated new cluster ID: 508fe791e2964f5dac7ad1c9eda997eb
I20260929 15:56:50.443389 5880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:50.455169 5880 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:50.457055 5880 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:50.469601 5880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0852f6f706bd4356bb978956365bfa96: Generated new TSK 0
I20260929 15:56:50.470649 5880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:50.482981 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 5797
I20260929 15:56:50.484519 5860 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:50.577226 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 5797
2026-09-29T15:56:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2022 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:50 dist-test-slave-c51v krb5kdc[5898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:50 dist-test-slave-c51v krb5kdc[5898](info): set up 2 sockets
Sep 29 15:56:50 dist-test-slave-c51v krb5kdc[5898](info): commencing operation
krb5kdc: starting...
W20260929 15:56:51.689728 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:51 dist-test-slave-c51v krb5kdc[5898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:56:51.946177 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:45157
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:46489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:45157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:52.271389 5911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:52.271670 5911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:52.271836 5911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:52.278946 5911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:52.279062 5911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:52.279199 5911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:52.279299 5911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:52.279341 5911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:52.289301 5911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:46489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:45157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:45157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:52.290700 5911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:52.292071 5911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:52.303942 5916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:52.303879 5917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:52.303998 5919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:52.304848 5911 server_base.cc:1034] running on GCE node
I20260929 15:56:52.305668 5911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:52.307046 5911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:52.308279 5911 hybrid_clock.cc:648] HybridClock initialized: now 1790697412308226 us; error 67 us; skew 500 ppm
I20260929 15:56:52.308655 5911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:52 dist-test-slave-c51v krb5kdc[5898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697412, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:52.311757 5911 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:56:52.313294 5911 webserver.cc:460] Webserver started at http://127.15.196.254:44903/ using document root <none> and password file <none>
I20260929 15:56:52.313972 5911 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:52.314118 5911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:52.314424 5911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:52.317425 5911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "58cb2e8539f94cfbb261a065d4559694"
format_stamp: "Formatted at 2026-09-29 15:56:52 on dist-test-slave-c51v"
server_key: "02993df8d280bd20a3761fb7af03bd0b"
server_key_iv: "ed5a8ce22cac554dee292bd8b40f6aae"
server_key_version: "encryptionkey@0"
I20260929 15:56:52.318181 5911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "58cb2e8539f94cfbb261a065d4559694"
format_stamp: "Formatted at 2026-09-29 15:56:52 on dist-test-slave-c51v"
server_key: "02993df8d280bd20a3761fb7af03bd0b"
server_key_iv: "ed5a8ce22cac554dee292bd8b40f6aae"
server_key_version: "encryptionkey@0"
I20260929 15:56:52.323483 5911 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 15:56:52.326772 5926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:52.328044 5911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:52.328222 5911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "58cb2e8539f94cfbb261a065d4559694"
format_stamp: "Formatted at 2026-09-29 15:56:52 on dist-test-slave-c51v"
server_key: "02993df8d280bd20a3761fb7af03bd0b"
server_key_iv: "ed5a8ce22cac554dee292bd8b40f6aae"
server_key_version: "encryptionkey@0"
I20260929 15:56:52.328411 5911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:52.352157 5911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:52.356170 5911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:52.356462 5911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:52.374637 5911 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:45157
I20260929 15:56:52.374660 5977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:45157 every 8 connection(s)
I20260929 15:56:52.376031 5911 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:52.379839 5978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:52.379997 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 5911
I20260929 15:56:52.380168 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:52.380733 16147 external_mini_cluster.cc:1428] Setting key 28b317d2f8aa970a895c359d85299721
I20260929 15:56:52.395165 5981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:52.382545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45397 (local address 127.15.196.254:45157)
0929 15:56:52.383364 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:52.383379 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:52.383439 (+ 60us) server_negotiation.cc:407] Connection header received
0929 15:56:52.385116 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:52.385143 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:52.385686 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:52.386124 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:52.386669 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:52.388744 (+ 2075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:52.390029 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:52.391040 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:52.391229 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:52.392390 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:52.392408 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:52.392425 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:52.393166 (+ 741us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:52.393271 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:52.393307 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:52.393465 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:52.393644 (+ 179us) server_negotiation.cc:299] Negotiation successful
0929 15:56:52.393888 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":337,"threads_started":1}
I20260929 15:56:52.395681 5978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694: Bootstrap starting.
I20260929 15:56:52.398813 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:52.398940 5978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:52.400189 5978 log.cc:824] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:52.403558 5978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694: No bootstrap required, opened a new log
I20260929 15:56:52.410822 5981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:52.404080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42811 (local address 127.15.196.254:45157)
0929 15:56:52.404266 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:52.404272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:52.404765 (+ 493us) server_negotiation.cc:407] Connection header received
0929 15:56:52.404912 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:52.404917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:52.404996 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:52.405104 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:52.405599 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:52.406750 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:52.408123 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:52.408889 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:52.409022 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:52.410053 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:52.410060 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:52.410064 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:52.410085 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:52.410129 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:52.410150 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:52.410243 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:52.410371 (+ 128us) server_negotiation.cc:299] Negotiation successful
0929 15:56:52.410466 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 15:56:52.413527 5978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58cb2e8539f94cfbb261a065d4559694" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45157 } }
I20260929 15:56:52.413918 5978 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:52.414057 5978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58cb2e8539f94cfbb261a065d4559694, State: Initialized, Role: FOLLOWER
I20260929 15:56:52.414623 5978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58cb2e8539f94cfbb261a065d4559694" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45157 } }
I20260929 15:56:52.414912 5978 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:52.415067 5978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:52.415237 5978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:52.417935 5978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58cb2e8539f94cfbb261a065d4559694" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45157 } }
I20260929 15:56:52.418604 5978 leader_election.cc:302] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58cb2e8539f94cfbb261a065d4559694; no voters:
I20260929 15:56:52.419340 5978 leader_election.cc:288] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:52.419456 5988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:52.420190 5988 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [term 1 LEADER]: Becoming Leader. State: Replica: 58cb2e8539f94cfbb261a065d4559694, State: Running, Role: LEADER
I20260929 15:56:52.420828 5988 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58cb2e8539f94cfbb261a065d4559694" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45157 } }
I20260929 15:56:52.421600 5978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:52.423789 5990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58cb2e8539f94cfbb261a065d4559694" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58cb2e8539f94cfbb261a065d4559694" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45157 } } }
I20260929 15:56:52.424242 5990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:52.423777 5989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58cb2e8539f94cfbb261a065d4559694. Latest consensus state: current_term: 1 leader_uuid: "58cb2e8539f94cfbb261a065d4559694" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58cb2e8539f94cfbb261a065d4559694" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 45157 } } }
I20260929 15:56:52.425463 5989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694 [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:52.428579 5995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:52.438902 5995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:52.449549 5995 catalog_manager.cc:1347] Generated new cluster ID: 7ee7f9c55d02498094a699ccf3e129eb
I20260929 15:56:52.449678 5995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:52.466001 5995 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:52.467936 5995 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:52.476310 5995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58cb2e8539f94cfbb261a065d4559694: Generated new TSK 0
I20260929 15:56:52.477303 5995 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:52.488243 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 5911
W20260929 15:56:52.489478 5973 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45397
W20260929 15:56:52.490209 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 5911: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:56:52.490370 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 5911
2026-09-29T15:56:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1912 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:52 dist-test-slave-c51v krb5kdc[6009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:52 dist-test-slave-c51v krb5kdc[6009](info): set up 2 sockets
Sep 29 15:56:52 dist-test-slave-c51v krb5kdc[6009](info): commencing operation
krb5kdc: starting...
W20260929 15:56:53.609870 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:53 dist-test-slave-c51v krb5kdc[6009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:56:53.870263 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:36593
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:44279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:36593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:54.202930 6022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:54.203231 6022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:54.203395 6022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:54.210850 6022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:54.210970 6022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:54.211017 6022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:54.211079 6022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:54.211123 6022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:54.221200 6022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:44279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:36593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:36593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:54.222507 6022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:54.223948 6022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:54.235864 6027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:54.235826 6028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:54.235905 6030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:54.237151 6022 server_base.cc:1034] running on GCE node
I20260929 15:56:54.237995 6022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:54.239464 6022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:54.240682 6022 hybrid_clock.cc:648] HybridClock initialized: now 1790697414240645 us; error 25 us; skew 500 ppm
I20260929 15:56:54.240996 6022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:54 dist-test-slave-c51v krb5kdc[6009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697414, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:54.244094 6022 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:56:54.245529 6022 webserver.cc:460] Webserver started at http://127.15.196.254:45153/ using document root <none> and password file <none>
I20260929 15:56:54.246156 6022 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:54.246253 6022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:54.246495 6022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:54.249439 6022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "32d262377c7b4a039773a623ff3eca1f"
format_stamp: "Formatted at 2026-09-29 15:56:54 on dist-test-slave-c51v"
server_key: "4eafe9dc8237795f68425bb672d126ab"
server_key_iv: "353ad343504d8e69ce942b3d6edbdb25"
server_key_version: "encryptionkey@0"
I20260929 15:56:54.250167 6022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "32d262377c7b4a039773a623ff3eca1f"
format_stamp: "Formatted at 2026-09-29 15:56:54 on dist-test-slave-c51v"
server_key: "4eafe9dc8237795f68425bb672d126ab"
server_key_iv: "353ad343504d8e69ce942b3d6edbdb25"
server_key_version: "encryptionkey@0"
I20260929 15:56:54.255522 6022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 15:56:54.258776 6037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:54.260123 6022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:54.260262 6022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "32d262377c7b4a039773a623ff3eca1f"
format_stamp: "Formatted at 2026-09-29 15:56:54 on dist-test-slave-c51v"
server_key: "4eafe9dc8237795f68425bb672d126ab"
server_key_iv: "353ad343504d8e69ce942b3d6edbdb25"
server_key_version: "encryptionkey@0"
I20260929 15:56:54.260425 6022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:54.266206 6022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:54.270023 6022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:54.270265 6022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:54.288901 6022 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:36593
I20260929 15:56:54.288923 6089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:36593 every 8 connection(s)
I20260929 15:56:54.290380 6022 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:54.294166 6090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:54.295136 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 6022
I20260929 15:56:54.295346 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:54.295922 16147 external_mini_cluster.cc:1428] Setting key 6485c3f6a81d53754268719c58fb0c81
I20260929 15:56:54.303889 6093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:54.297711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50489 (local address 127.15.196.254:36593)
0929 15:56:54.298552 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:54.298564 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:54.298612 (+ 48us) server_negotiation.cc:407] Connection header received
0929 15:56:54.300493 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:54.300516 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:54.300912 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:54.301267 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:54.302305 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:54.302311 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:54.302324 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:54.302338 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:54.302394 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:54.302418 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:54.302577 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:54.302698 (+ 121us) server_negotiation.cc:299] Negotiation successful
0929 15:56:54.302937 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":376,"threads_started":1}
I20260929 15:56:54.307229 6090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f: Bootstrap starting.
I20260929 15:56:54.307492 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:54.310972 6090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:54.312366 6090 log.cc:824] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:54.315128 6093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:54.312679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52345 (local address 127.15.196.254:36593)
0929 15:56:54.312924 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:54.312930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:54.313279 (+ 349us) server_negotiation.cc:407] Connection header received
0929 15:56:54.313391 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:54.313398 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:54.313463 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:54.313580 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:54.314477 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:54.314481 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:54.314484 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 15:56:54.314497 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:54.314533 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 15:56:54.314548 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:54.314632 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:54.314783 (+ 151us) server_negotiation.cc:299] Negotiation successful
0929 15:56:54.314835 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 15:56:54.315900 6090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f: No bootstrap required, opened a new log
I20260929 15:56:54.323052 6090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d262377c7b4a039773a623ff3eca1f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36593 } }
I20260929 15:56:54.323477 6090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 15:56:54.323607 6090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d262377c7b4a039773a623ff3eca1f, State: Initialized, Role: FOLLOWER
I20260929 15:56:54.324182 6090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d262377c7b4a039773a623ff3eca1f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36593 } }
I20260929 15:56:54.324421 6090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 15:56:54.324570 6090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 15:56:54.324746 6090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:54.326946 6090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d262377c7b4a039773a623ff3eca1f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36593 } }
I20260929 15:56:54.327512 6090 leader_election.cc:302] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32d262377c7b4a039773a623ff3eca1f; no voters:
I20260929 15:56:54.328099 6090 leader_election.cc:288] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 15:56:54.328224 6100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:54.328876 6100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [term 1 LEADER]: Becoming Leader. State: Replica: 32d262377c7b4a039773a623ff3eca1f, State: Running, Role: LEADER
I20260929 15:56:54.329453 6100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d262377c7b4a039773a623ff3eca1f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36593 } }
I20260929 15:56:54.330171 6090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:54.332346 6101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32d262377c7b4a039773a623ff3eca1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d262377c7b4a039773a623ff3eca1f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36593 } } }
I20260929 15:56:54.332679 6101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:54.334275 6107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:56:54.334200 6102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32d262377c7b4a039773a623ff3eca1f. Latest consensus state: current_term: 1 leader_uuid: "32d262377c7b4a039773a623ff3eca1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32d262377c7b4a039773a623ff3eca1f" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 36593 } } }
I20260929 15:56:54.334513 6102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:54.340055 6107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:54.350356 6107 catalog_manager.cc:1347] Generated new cluster ID: 6d030d41cdea4f24bf1360247a0f56d5
I20260929 15:56:54.350469 6107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:54.374531 6107 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:54.374987 6107 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:54.386674 6107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32d262377c7b4a039773a623ff3eca1f: Generated new TSK 0
I20260929 15:56:54.387662 6107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:54.402792 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 6022
W20260929 15:56:54.403898 6085 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50489
W20260929 15:56:54.404615 16147 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 6022: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 15:56:54.404738 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 6022
2026-09-29T15:56:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1914 ms)
[----------] 18 tests from AuthTokenIssuingTest (35025 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:54 dist-test-slave-c51v krb5kdc[6122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:54 dist-test-slave-c51v krb5kdc[6122](info): set up 2 sockets
Sep 29 15:56:54 dist-test-slave-c51v krb5kdc[6122](info): commencing operation
krb5kdc: starting...
W20260929 15:56:55.534693 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:56:55 dist-test-slave-c51v krb5kdc[6122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:56:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:56:55.801326 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:35789
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:38519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:35789,127.15.196.253:39899,127.15.196.252:36925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:56:56.137064 6135 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 15:56:56.137468 6135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:56.137599 6135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:56.137740 6135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:56:56.145192 6135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:56.145324 6135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:56.145447 6135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:56.145538 6135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:56.145627 6135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:56.155273 6135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:38519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:35789,127.15.196.253:39899,127.15.196.252:36925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:35789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:56.156669 6135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:56.158025 6135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:56.169540 6140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:56.169466 6141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:56.169516 6143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:56.170324 6135 server_base.cc:1034] running on GCE node
I20260929 15:56:56.171051 6135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:56.172467 6135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:56.173710 6135 hybrid_clock.cc:648] HybridClock initialized: now 1790697416173661 us; error 47 us; skew 500 ppm
I20260929 15:56:56.174027 6135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:56 dist-test-slave-c51v krb5kdc[6122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697416, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:56.184011 6135 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:56:56.185462 6135 webserver.cc:460] Webserver started at http://127.15.196.254:46163/ using document root <none> and password file <none>
I20260929 15:56:56.186131 6135 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:56.186213 6135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:56.186477 6135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:56.189386 6135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "8c22b2ede2ad4e699d851ede8189d98b"
format_stamp: "Formatted at 2026-09-29 15:56:56 on dist-test-slave-c51v"
server_key: "050650ad20d568dd5420166b2fd93a84"
server_key_iv: "4373180d128925d18e3b2a7b5a0d0fb4"
server_key_version: "encryptionkey@0"
I20260929 15:56:56.190117 6135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "8c22b2ede2ad4e699d851ede8189d98b"
format_stamp: "Formatted at 2026-09-29 15:56:56 on dist-test-slave-c51v"
server_key: "050650ad20d568dd5420166b2fd93a84"
server_key_iv: "4373180d128925d18e3b2a7b5a0d0fb4"
server_key_version: "encryptionkey@0"
I20260929 15:56:56.195169 6135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 15:56:56.198732 6150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:56.200109 6135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:56.200294 6135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "8c22b2ede2ad4e699d851ede8189d98b"
format_stamp: "Formatted at 2026-09-29 15:56:56 on dist-test-slave-c51v"
server_key: "050650ad20d568dd5420166b2fd93a84"
server_key_iv: "4373180d128925d18e3b2a7b5a0d0fb4"
server_key_version: "encryptionkey@0"
I20260929 15:56:56.200527 6135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:56.210839 6135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:56.214612 6135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:56.214879 6135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:56.233122 6135 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:35789
I20260929 15:56:56.233146 6202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:35789 every 8 connection(s)
I20260929 15:56:56.234524 6135 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:56:56.237309 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 6135
I20260929 15:56:56.237529 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:56:56.238332 16147 external_mini_cluster.cc:1428] Setting key 2f2c7a870aff42f77e0a3c4105f310ae
I20260929 15:56:56.238701 6203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:56.248796 6203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:56:56 dist-test-slave-c51v krb5kdc[6122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697416, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.196.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.253@KRBTEST.COM" created.
I20260929 15:56:56.273664 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.252237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:35789 (local address 127.0.0.1:41708)
0929 15:56:56.253230 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.253292 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:56.255175 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:56.256188 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:56.256194 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:56.256789 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:56.257117 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:56.257133 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:56.259476 (+ 2343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:56.259480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:56.260546 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:56.260557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:56.261894 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:56.261899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:56.262015 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:56.262773 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:56.262808 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:56.265561 (+ 2753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:56.269286 (+ 3725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:56.269296 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:56.269306 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:56.269777 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:56.270187 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:56.270194 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:56.270200 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:56.270347 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:56.271113 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:56.271125 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:56.271423 (+ 298us) client_negotiation.cc:769] Sending connection context
0929 15:56:56.271669 (+ 246us) client_negotiation.cc:240] Negotiation successful
0929 15:56:56.272246 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":135,"thread_start_us":378,"threads_started":1}
I20260929 15:56:56.273698 6207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.252236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41708 (local address 127.15.196.254:35789)
0929 15:56:56.253893 (+ 1657us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:56.253906 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:56.253960 (+ 54us) server_negotiation.cc:407] Connection header received
0929 15:56:56.255619 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:56.255630 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:56.255943 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:56.256165 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:56.257333 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:56.259326 (+ 1993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:56.260688 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:56.261676 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:56.261871 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:56.265818 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:56.265829 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:56.265847 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:56.265889 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:56.269160 (+ 3271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:56.269925 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:56.269932 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:56.269935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:56.270073 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:56.270461 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:56.270466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:56.270469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:56.270979 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:56.271193 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:56.271786 (+ 593us) server_negotiation.cc:299] Negotiation successful
0929 15:56:56.272124 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1448,"spinlock_wait_cycles":40192,"thread_start_us":1239,"threads_started":1}
I20260929 15:56:56.276800 6203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } has no permanent_uuid. Determining permanent_uuid...
I20260929 15:56:56.277895 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.277502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:39899 (local address 127.0.0.1:50844)
0929 15:56:56.277672 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.277787 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:39899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260929 15:56:56.278261 6154 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.196.253:39899: connect: Connection refused (error 111)
I20260929 15:56:56.279969 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.279628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:39899 (local address 127.0.0.1:50854)
0929 15:56:56.279768 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.279835 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:39899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260929 15:56:56.280287 6203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.253:39899: Network error: Client connection negotiation failed: client connection to 127.15.196.253:39899: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.253.keytab.
Entry for principal kudu/127.15.196.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.253.keytab.
I20260929 15:56:56.302141 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.253:39899
--webserver_interface=127.15.196.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.253.keytab
--principal=kudu/127.15.196.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:38519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:35789,127.15.196.253:39899,127.15.196.252:36925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:56:56.329661 6203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } attempt: 1
I20260929 15:56:56.330881 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.330423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:39899 (local address 127.0.0.1:50860)
0929 15:56:56.330627 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.330749 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:39899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260929 15:56:56.332056 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.331678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:39899 (local address 127.0.0.1:50862)
0929 15:56:56.331859 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.331927 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:39899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260929 15:56:56.332298 6203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.253:39899: Network error: Client connection negotiation failed: client connection to 127.15.196.253:39899: connect: Connection refused (error 111)
I20260929 15:56:56.400568 6203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } attempt: 2
I20260929 15:56:56.401947 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.401400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:39899 (local address 127.0.0.1:50878)
0929 15:56:56.401685 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.401792 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:39899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260929 15:56:56.403093 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.402685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:39899 (local address 127.0.0.1:50880)
0929 15:56:56.402879 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.402962 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:39899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260929 15:56:56.403363 6203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.253:39899: Network error: Client connection negotiation failed: client connection to 127.15.196.253:39899: connect: Connection refused (error 111)
I20260929 15:56:56.494691 6203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } attempt: 3
I20260929 15:56:56.496053 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.495567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:39899 (local address 127.0.0.1:50888)
0929 15:56:56.495791 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.495900 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:39899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260929 15:56:56.497308 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.496897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:39899 (local address 127.0.0.1:50904)
0929 15:56:56.497060 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.497141 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:39899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260929 15:56:56.497586 6203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.253:39899: Network error: Client connection negotiation failed: client connection to 127.15.196.253:39899: connect: Connection refused (error 111)
W20260929 15:56:56.636435 6210 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 15:56:56.636786 6210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:56.636909 6210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:56.637049 6210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260929 15:56:56.640841 6203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } attempt: 4
I20260929 15:56:56.642102 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.641502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:39899 (local address 127.0.0.1:50920)
0929 15:56:56.641846 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.641961 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:39899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260929 15:56:56.643136 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.642776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:39899 (local address 127.0.0.1:50926)
0929 15:56:56.642929 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.643017 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:39899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260929 15:56:56.643399 6203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.253:39899: Network error: Client connection negotiation failed: client connection to 127.15.196.253:39899: connect: Connection refused (error 111)
W20260929 15:56:56.644634 6210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:56.644717 6210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:56.644804 6210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:56.644893 6210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:56.645008 6210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:56.654901 6210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:38519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:35789,127.15.196.253:39899,127.15.196.252:36925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.253.keytab
--principal=kudu/127.15.196.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.253:39899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.196.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:56.656353 6210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:56.657646 6210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:56.669154 6218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:56.669234 6215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:56.669201 6216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:56.670295 6210 server_base.cc:1034] running on GCE node
I20260929 15:56:56.671020 6210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:56.672411 6210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:56.673652 6210 hybrid_clock.cc:648] HybridClock initialized: now 1790697416673597 us; error 32 us; skew 500 ppm
I20260929 15:56:56.673985 6210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:56 dist-test-slave-c51v krb5kdc[6122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697416, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:56.677162 6210 init.cc:377] Logged in from keytab as kudu/127.15.196.253@KRBTEST.COM (short username kudu)
I20260929 15:56:56.678603 6210 webserver.cc:460] Webserver started at http://127.15.196.253:40797/ using document root <none> and password file <none>
I20260929 15:56:56.679340 6210 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:56.679478 6210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:56.679800 6210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:56.682746 6210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/data/instance:
uuid: "870fe458763541cf99ca3a76b2ae2bef"
format_stamp: "Formatted at 2026-09-29 15:56:56 on dist-test-slave-c51v"
server_key: "a75611479b79d575ca6c97d9bb207e2a"
server_key_iv: "4b024c6f26935b90a67cc006aabc0a60"
server_key_version: "encryptionkey@0"
I20260929 15:56:56.683629 6210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/wal/instance:
uuid: "870fe458763541cf99ca3a76b2ae2bef"
format_stamp: "Formatted at 2026-09-29 15:56:56 on dist-test-slave-c51v"
server_key: "a75611479b79d575ca6c97d9bb207e2a"
server_key_iv: "4b024c6f26935b90a67cc006aabc0a60"
server_key_version: "encryptionkey@0"
I20260929 15:56:56.688804 6210 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 15:56:56.692224 6225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:56.693500 6210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 15:56:56.693684 6210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/wal
uuid: "870fe458763541cf99ca3a76b2ae2bef"
format_stamp: "Formatted at 2026-09-29 15:56:56 on dist-test-slave-c51v"
server_key: "a75611479b79d575ca6c97d9bb207e2a"
server_key_iv: "4b024c6f26935b90a67cc006aabc0a60"
server_key_version: "encryptionkey@0"
I20260929 15:56:56.693904 6210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:56.740178 6210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:56.744465 6210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:56.744753 6210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:56.763118 6210 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.253:39899
I20260929 15:56:56.763137 6277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.253:39899 every 8 connection(s)
I20260929 15:56:56.764530 6210 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/data/info.pb
I20260929 15:56:56.768128 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 6210
I20260929 15:56:56.768322 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-1/wal/instance
I20260929 15:56:56.768882 16147 external_mini_cluster.cc:1428] Setting key 8d7c3b6db153ff5fe046bdf3910a5400
I20260929 15:56:56.768924 6278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:56.778703 6278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:56:56 dist-test-slave-c51v krb5kdc[6122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697416, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.253@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.196.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.252@KRBTEST.COM" created.
I20260929 15:56:56.801056 6281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.781685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41718 (local address 127.15.196.254:35789)
0929 15:56:56.782215 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:56.782224 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:56.783195 (+ 971us) server_negotiation.cc:407] Connection header received
0929 15:56:56.785149 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:56.785157 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:56.785269 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:56.785407 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:56.786969 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:56.788191 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:56.789697 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:56.790458 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:56.790604 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:56.794448 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:56.794457 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:56.794465 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:56.794529 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:56.797684 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:56.798569 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:56.798574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:56.798576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:56.798716 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:56.799193 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:56.799199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:56.799203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:56.799576 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:56.799731 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:56.800456 (+ 725us) server_negotiation.cc:299] Negotiation successful
0929 15:56:56.800676 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":249,"threads_started":1}
I20260929 15:56:56.802028 6282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.782250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:35789 (local address 127.0.0.1:41718)
0929 15:56:56.783018 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.783067 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:56.784866 (+ 1799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:56.785609 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:56.785617 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:56.786120 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:56.786741 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:56.786758 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:56.788351 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:56.788355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:56.789549 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:56.789560 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:56.790666 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:56.790671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:56.790864 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:56.791717 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:56.791752 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:56.794200 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:56.797810 (+ 3610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:56.797818 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:56.797830 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:56.798446 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:56.798839 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:56.798846 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:56.798849 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:56.799084 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:56.799722 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:56.799730 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:56.800081 (+ 351us) client_negotiation.cc:769] Sending connection context
0929 15:56:56.800358 (+ 277us) client_negotiation.cc:240] Negotiation successful
0929 15:56:56.800872 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":338,"threads_started":1}
I20260929 15:56:56.803824 6278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:56:56 dist-test-slave-c51v krb5kdc[6122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697416, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.253@KRBTEST.COM for kudu/127.15.196.253@KRBTEST.COM
I20260929 15:56:56.818831 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.804868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50940 (local address 127.15.196.253:39899)
0929 15:56:56.805590 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:56.805596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:56.805619 (+ 23us) server_negotiation.cc:407] Connection header received
0929 15:56:56.805659 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:56.805664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:56.805835 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:56.805990 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:56.806695 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:56.807823 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:56.808900 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:56.809421 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:56.809550 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:56.813273 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:56.813282 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:56.813296 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:56.813322 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:56.816041 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:56.816807 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:56.816813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:56.816815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:56.816926 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:56.817340 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:56.817343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:56.817345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:56.817649 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:56.817782 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:56.818266 (+ 484us) server_negotiation.cc:299] Negotiation successful
0929 15:56:56.818549 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":575,"thread_start_us":392,"threads_started":1}
I20260929 15:56:56.818917 6282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.804622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:39899 (local address 127.0.0.1:50940)
0929 15:56:56.804808 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.804826 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:56.804937 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:56.805980 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:56.805984 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:56.806392 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:56.806526 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:56.806534 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:56.807946 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:56.807952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:56.808768 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:56.808778 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:56.809558 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:56.809563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:56.809776 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:56.810382 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:56.810399 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:56.813079 (+ 2680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:56.816179 (+ 3100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:56.816187 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:56.816194 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:56.816682 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:56.817039 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:56.817047 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:56.817052 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:56.817238 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:56.817773 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:56.817780 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:56.817952 (+ 172us) client_negotiation.cc:769] Sending connection context
0929 15:56:56.818104 (+ 152us) client_negotiation.cc:240] Negotiation successful
0929 15:56:56.818430 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":17}
I20260929 15:56:56.821261 6278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } has no permanent_uuid. Determining permanent_uuid...
I20260929 15:56:56.822487 6282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.821964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41604)
0929 15:56:56.822172 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.822291 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260929 15:56:56.822767 6227 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
I20260929 15:56:56.824270 6282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.823966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41614)
0929 15:56:56.824100 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.824174 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260929 15:56:56.824482 6278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:36925: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.252.keytab.
Entry for principal kudu/127.15.196.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.252.keytab.
I20260929 15:56:56.831212 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.252:36925
--webserver_interface=127.15.196.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.252.keytab
--principal=kudu/127.15.196.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:38519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:35789,127.15.196.253:39899,127.15.196.252:36925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:56:56.873943 6278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } attempt: 1
I20260929 15:56:56.875200 6282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.874625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41618)
0929 15:56:56.874860 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.875017 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260929 15:56:56.876278 6282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.875871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41624)
0929 15:56:56.876054 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.876149 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260929 15:56:56.876498 6278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:36925: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
I20260929 15:56:56.942649 6203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } attempt: 5
I20260929 15:56:56.944778 6278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } attempt: 2
I20260929 15:56:56.945935 6282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.945361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41634)
0929 15:56:56.945583 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.945713 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 15:56:56.948343 6282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.947758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41640)
0929 15:56:56.948007 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.948109 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
W20260929 15:56:56.948799 6278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:36925: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Sep 29 15:56:56 dist-test-slave-c51v krb5kdc[6122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697416, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for kudu/127.15.196.253@KRBTEST.COM
I20260929 15:56:56.960803 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.944007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50948 (local address 127.15.196.253:39899)
0929 15:56:56.944268 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:56.944275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:56.944301 (+ 26us) server_negotiation.cc:407] Connection header received
0929 15:56:56.944394 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:56.944401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:56.944508 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:56.944676 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:56.947457 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:56.948670 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:56.950417 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:56.950968 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:56.951144 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:56.955216 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:56.955222 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:56.955226 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:56.955269 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:56.958095 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:56.958790 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:56.958797 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:56.958801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:56.958929 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:56.959334 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:56.959339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:56.959342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:56.959634 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:56.959791 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:56.960249 (+ 458us) server_negotiation.cc:299] Negotiation successful
0929 15:56:56.960477 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260929 15:56:56.960814 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.943600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:39899 (local address 127.0.0.1:50948)
0929 15:56:56.943886 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.943933 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:56.944120 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:56.946297 (+ 2177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:56.946304 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:56.946949 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:56.947247 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:56.947266 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:56.948863 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:56.948868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:56.950207 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:56.950222 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:56.951168 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:56.951176 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:56.951478 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:56.952186 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:56.952221 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:56.954987 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:56.958263 (+ 3276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:56.958268 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:56.958273 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:56.958666 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:56.959053 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:56.959058 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:56.959062 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:56.959200 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:56.959753 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:56.959758 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:56.959973 (+ 215us) client_negotiation.cc:769] Sending connection context
0929 15:56:56.960146 (+ 173us) client_negotiation.cc:240] Negotiation successful
0929 15:56:56.960455 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260929 15:56:56.961792 6203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } has no permanent_uuid. Determining permanent_uuid...
I20260929 15:56:56.962811 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.962423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41650)
0929 15:56:56.962599 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.962683 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260929 15:56:56.963744 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:56.963437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41664)
0929 15:56:56.963583 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:56.963654 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 15:56:56.963969 6203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:36925: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
I20260929 15:56:57.015206 6203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } attempt: 1
I20260929 15:56:57.016605 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.015930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41670)
0929 15:56:57.016200 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:57.016351 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260929 15:56:57.017831 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.017357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41680)
0929 15:56:57.017542 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:57.017653 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260929 15:56:57.018090 6203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:36925: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
I20260929 15:56:57.040040 6278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } attempt: 3
I20260929 15:56:57.041392 6282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.040882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41688)
0929 15:56:57.041079 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:57.041171 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260929 15:56:57.042588 6282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.042176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41698)
0929 15:56:57.042416 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:57.042494 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
W20260929 15:56:57.042840 6278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:36925: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
I20260929 15:56:57.086349 6203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } attempt: 2
I20260929 15:56:57.087829 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.087121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41704)
0929 15:56:57.087451 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:57.087595 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260929 15:56:57.089026 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.088600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41706)
0929 15:56:57.088771 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:57.088870 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260929 15:56:57.089291 6203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:36925: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
W20260929 15:56:57.185591 6286 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 15:56:57.185940 6286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:56:57.186055 6286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:56:57.186198 6286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260929 15:56:57.186084 6278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } attempt: 4
I20260929 15:56:57.187330 6282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.186774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41708)
0929 15:56:57.187001 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:57.187108 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260929 15:56:57.188299 6282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.187984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41724)
0929 15:56:57.188125 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:57.188203 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 15:56:57.188572 6278 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:36925: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
W20260929 15:56:57.194180 6286 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:56:57.194298 6286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:56:57.194423 6286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:56:57.194519 6286 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:56:57.194639 6286 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:56:57.195518 6203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } attempt: 3
I20260929 15:56:57.196607 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.196069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41732)
0929 15:56:57.196287 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:57.196395 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260929 15:56:57.197799 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.197351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41744)
0929 15:56:57.197550 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:57.197646 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260929 15:56:57.198076 6203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:36925: Network error: Client connection negotiation failed: client connection to 127.15.196.252:36925: connect: Connection refused (error 111)
I20260929 15:56:57.206024 6286 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:38519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:35789,127.15.196.253:39899,127.15.196.252:36925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.252.keytab
--principal=kudu/127.15.196.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.252:36925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.196.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:56:57.207561 6286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:56:57.209005 6286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:56:57.221725 6294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:57.221763 6292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:56:57.221778 6291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:56:57.222581 6286 server_base.cc:1034] running on GCE node
I20260929 15:56:57.223558 6286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:56:57.225098 6286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:56:57.226375 6286 hybrid_clock.cc:648] HybridClock initialized: now 1790697417226311 us; error 49 us; skew 500 ppm
I20260929 15:56:57.226732 6286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:56:57 dist-test-slave-c51v krb5kdc[6122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697417, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:56:57.230329 6286 init.cc:377] Logged in from keytab as kudu/127.15.196.252@KRBTEST.COM (short username kudu)
I20260929 15:56:57.231956 6286 webserver.cc:460] Webserver started at http://127.15.196.252:39765/ using document root <none> and password file <none>
I20260929 15:56:57.232692 6286 fs_manager.cc:359] Metadata directory not provided
I20260929 15:56:57.232823 6286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:56:57.233212 6286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:56:57.236428 6286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/data/instance:
uuid: "977aabd1762642a79d0adb753b8d36f4"
format_stamp: "Formatted at 2026-09-29 15:56:57 on dist-test-slave-c51v"
server_key: "00c3cce3c1f7dd87480246267a70390a"
server_key_iv: "7c016a116ad1d7b72edbe74ce5435dff"
server_key_version: "encryptionkey@0"
I20260929 15:56:57.237272 6286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/wal/instance:
uuid: "977aabd1762642a79d0adb753b8d36f4"
format_stamp: "Formatted at 2026-09-29 15:56:57 on dist-test-slave-c51v"
server_key: "00c3cce3c1f7dd87480246267a70390a"
server_key_iv: "7c016a116ad1d7b72edbe74ce5435dff"
server_key_version: "encryptionkey@0"
I20260929 15:56:57.243181 6286 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260929 15:56:57.246786 6301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:56:57.248168 6286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 15:56:57.248327 6286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/wal
uuid: "977aabd1762642a79d0adb753b8d36f4"
format_stamp: "Formatted at 2026-09-29 15:56:57 on dist-test-slave-c51v"
server_key: "00c3cce3c1f7dd87480246267a70390a"
server_key_iv: "7c016a116ad1d7b72edbe74ce5435dff"
server_key_version: "encryptionkey@0"
I20260929 15:56:57.248530 6286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:56:57.273080 6286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:56:57.277871 6286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:56:57.278201 6286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:56:57.298625 6286 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.252:36925
I20260929 15:56:57.298647 6352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.252:36925 every 8 connection(s)
I20260929 15:56:57.300171 6286 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/data/info.pb
I20260929 15:56:57.304531 6353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:56:57.306550 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 6286
I20260929 15:56:57.306738 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697059131395-16147-0/minicluster-data/master-2/wal/instance
I20260929 15:56:57.307304 16147 external_mini_cluster.cc:1428] Setting key 2ae9e6c9ebddf7ad62286c0c505a1320
I20260929 15:56:57.312582 6353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:56:57 dist-test-slave-c51v krb5kdc[6122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.252@KRBTEST.COM
Sep 29 15:56:57 dist-test-slave-c51v krb5kdc[6122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697417, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.252@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:56:57.330331 6357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.314968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41722 (local address 127.15.196.254:35789)
0929 15:56:57.315515 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.315519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.316111 (+ 592us) server_negotiation.cc:407] Connection header received
0929 15:56:57.316327 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.316331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.316409 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.316538 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:57.317603 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.318300 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.319691 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.320066 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.320200 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.323743 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:57.323752 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:57.323759 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:57.323800 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:57.327036 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.327927 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.327931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.327934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.328086 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.328570 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.328575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.328578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.328919 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:57.329056 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:57.329743 (+ 687us) server_negotiation.cc:299] Negotiation successful
0929 15:56:57.329944 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":341,"threads_started":1}
I20260929 15:56:57.331153 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.309340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41746 (local address 127.15.196.252:36925)
0929 15:56:57.310153 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.310167 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.310216 (+ 49us) server_negotiation.cc:407] Connection header received
0929 15:56:57.311992 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.312017 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.312418 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.312818 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:57.314292 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.316424 (+ 2132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.317703 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.318311 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.318540 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.322862 (+ 4322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:57.322871 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:57.322881 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:57.322910 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:57.326510 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.327412 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.327418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.327420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.327578 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.328062 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.328066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.328069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.328826 (+ 757us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:57.329147 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:57.329533 (+ 386us) server_negotiation.cc:299] Negotiation successful
0929 15:56:57.329972 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"spinlock_wait_cycles":14080,"thread_start_us":350,"threads_started":1}
I20260929 15:56:57.331197 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.314991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:35789 (local address 127.0.0.1:41722)
0929 15:56:57.315972 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:57.316005 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:57.316185 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:57.316595 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:57.316602 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:57.317215 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:57.317455 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:57.317464 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.318451 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.318458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:57.319530 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:57.319541 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.320282 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.320289 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:57.320439 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.321058 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:57.321078 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:57.323509 (+ 2431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:57.327178 (+ 3669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:57.327190 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:57.327202 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:57.327789 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:57.328219 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:57.328222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:57.328225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:57.328386 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:57.329068 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:57.329076 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:57.329331 (+ 255us) client_negotiation.cc:769] Sending connection context
0929 15:56:57.329605 (+ 274us) client_negotiation.cc:240] Negotiation successful
0929 15:56:57.330162 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"mutex_wait_us":165,"thread_start_us":669,"threads_started":1}
I20260929 15:56:57.332820 6353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:56:57 dist-test-slave-c51v krb5kdc[6122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697417, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.252@KRBTEST.COM for kudu/127.15.196.253@KRBTEST.COM
Sep 29 15:56:57 dist-test-slave-c51v krb5kdc[6122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:56:57.347494 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.333618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:39899 (local address 127.0.0.1:50958)
0929 15:56:57.333866 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:57.333884 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:57.333990 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:57.334295 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:57.334299 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:57.334687 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:57.334865 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:57.334876 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.336321 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.336325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:57.337664 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:57.337677 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.338597 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.338601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:57.338753 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.339358 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:57.339376 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:57.341629 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:57.345156 (+ 3527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:57.345166 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:57.345169 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:57.345528 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:57.345939 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:57.345946 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:57.345953 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:57.346130 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:57.346751 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:57.346754 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:57.346880 (+ 126us) client_negotiation.cc:769] Sending connection context
0929 15:56:57.346990 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 15:56:57.347189 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260929 15:56:57.347805 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.333597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50958 (local address 127.15.196.253:39899)
0929 15:56:57.333798 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.333803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.333954 (+ 151us) server_negotiation.cc:407] Connection header received
0929 15:56:57.334093 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.334097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.334157 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.334234 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:57.335045 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.336158 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.337811 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.338431 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.338587 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.341829 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:57.341838 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:57.341844 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:57.341889 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:57.345001 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.345641 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.345646 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.345651 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.345818 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.346240 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.346247 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.346252 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.346639 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:57.346789 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:57.347131 (+ 342us) server_negotiation.cc:299] Negotiation successful
0929 15:56:57.347380 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 15:56:57.348825 6353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } has no permanent_uuid. Determining permanent_uuid...
I20260929 15:56:57.350996 6357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.334667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41730 (local address 127.15.196.254:35789)
0929 15:56:57.334845 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.334851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.334868 (+ 17us) server_negotiation.cc:407] Connection header received
0929 15:56:57.334988 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.334992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.335067 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.335146 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:57.336646 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.337671 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.339179 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.339645 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.339770 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.344302 (+ 4532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:57.344309 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:57.344313 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:57.344353 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:57.347617 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.348431 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.348437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.348440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.348613 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.349111 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.349116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.349119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.349442 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:57.349740 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:57.350569 (+ 829us) server_negotiation.cc:299] Negotiation successful
0929 15:56:57.350732 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Sep 29 15:56:57 dist-test-slave-c51v krb5kdc[6122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697417, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.252@KRBTEST.COM for kudu/127.15.196.252@KRBTEST.COM
Sep 29 15:56:57 dist-test-slave-c51v krb5kdc[6122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.253@KRBTEST.COM
I20260929 15:56:57.363327 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.349633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41758)
0929 15:56:57.349789 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:57.349807 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:57.349898 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:57.350283 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:57.350286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:57.350699 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:57.350827 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:57.350835 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.352128 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.352133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:57.353299 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:57.353311 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.354000 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.354005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:57.354222 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.354791 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:57.354807 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:57.356766 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:57.360693 (+ 3927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:57.360698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:57.360703 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:57.361233 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:57.361621 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:57.361628 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:57.361634 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:57.361761 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:57.362349 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:57.362353 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:57.362503 (+ 150us) client_negotiation.cc:769] Sending connection context
0929 15:56:57.362616 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 15:56:57.362890 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260929 15:56:57.363682 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.349847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41758 (local address 127.15.196.252:36925)
0929 15:56:57.350027 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.350032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.350048 (+ 16us) server_negotiation.cc:407] Connection header received
0929 15:56:57.350091 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.350096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.350158 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.350261 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:57.350971 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.352030 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.353428 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.353846 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.353994 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.356942 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:57.356950 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:57.356955 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:57.356987 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:57.360542 (+ 3555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.361379 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.361383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.361387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.361516 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.361861 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.361868 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.361871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.362223 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:57.362373 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:57.362751 (+ 378us) server_negotiation.cc:299] Negotiation successful
0929 15:56:57.363084 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":60}
I20260929 15:56:57.367961 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.352782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50970 (local address 127.15.196.253:39899)
0929 15:56:57.352947 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.352953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.352970 (+ 17us) server_negotiation.cc:407] Connection header received
0929 15:56:57.353085 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.353090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.353158 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.353256 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:57.354383 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.355062 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.356344 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.356767 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.356897 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.362180 (+ 5283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:57.362187 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:57.362192 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:57.362229 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:57.365433 (+ 3204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.366213 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.366216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.366218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.366345 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.366745 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.366748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.366750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.366990 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:57.367194 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:57.367562 (+ 368us) server_negotiation.cc:299] Negotiation successful
0929 15:56:57.367722 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 15:56:57.369915 6353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4: Bootstrap starting.
I20260929 15:56:57.373006 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:56:57.373909 6353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:57.375188 6353 log.cc:824] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:57.375339 6203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } attempt: 4
I20260929 15:56:57.378971 6353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4: No bootstrap required, opened a new log
I20260929 15:56:57.376698 6177 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8c22b2ede2ad4e699d851ede8189d98b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41730
Sep 29 15:56:57 dist-test-slave-c51v krb5kdc[6122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697416, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for kudu/127.15.196.252@KRBTEST.COM
I20260929 15:56:57.388058 6353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } }
I20260929 15:56:57.388521 6353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977aabd1762642a79d0adb753b8d36f4, State: Initialized, Role: FOLLOWER
I20260929 15:56:57.389087 6353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } }
I20260929 15:56:57.390159 6360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } } }
I20260929 15:56:57.390597 6360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:56:57.391597 6353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:57.392474 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.375968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41768)
0929 15:56:57.376186 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:57.376208 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:57.376327 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:57.376720 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:57.376725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:57.377285 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:57.377513 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:57.377529 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.379219 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.379225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:57.380741 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:57.380759 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.381818 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.381826 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:57.382038 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.382685 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:57.382708 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:57.385583 (+ 2875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:57.389272 (+ 3689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:57.389278 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:57.389282 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:57.389878 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:57.390335 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:57.390354 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:57.390357 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:57.390541 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:57.391378 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:57.391383 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:57.391596 (+ 213us) client_negotiation.cc:769] Sending connection context
0929 15:56:57.391760 (+ 164us) client_negotiation.cc:240] Negotiation successful
0929 15:56:57.392053 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 15:56:57.393635 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.376285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41768 (local address 127.15.196.252:36925)
0929 15:56:57.376456 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.376461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.376476 (+ 15us) server_negotiation.cc:407] Connection header received
0929 15:56:57.376526 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.376529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.376581 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.376693 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:57.377701 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.379040 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.380897 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.381662 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.381803 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.385824 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:57.385829 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:57.385833 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:57.385893 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:57.389131 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.390027 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.390034 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.390042 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.390212 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.390681 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.390689 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.390692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.391135 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:57.391313 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:57.392787 (+ 1474us) server_negotiation.cc:299] Negotiation successful
0929 15:56:57.393076 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 15:56:57.399933 6203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b: Bootstrap starting.
W20260929 15:56:57.402420 6371 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 15:56:57.402559 6371 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 15:56:57.404114 6203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:57.405388 6203 log.cc:824] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:57.408676 6203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b: No bootstrap required, opened a new log
I20260929 15:56:57.399967 6286 master_runner.cc:186] Error getting master registration for 127.15.196.254:35789: OK, instance_id { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" instance_seqno: 1790697416227965 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 15:56:57.410531 6286 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 15:56:57.414530 6203 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } }
I20260929 15:56:57.415824 6203 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c22b2ede2ad4e699d851ede8189d98b, State: Initialized, Role: FOLLOWER
I20260929 15:56:57.416330 6203 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } }
I20260929 15:56:57.417338 6374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } } }
I20260929 15:56:57.417819 6374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:56:57.418504 6203 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:57.418587 6177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 15:56:57.418763 6177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:57.422889 6177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } }
I20260929 15:56:57.426085 6177 leader_election.cc:288] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [CANDIDATE]: Term 1 election: Requested vote from peers 870fe458763541cf99ca3a76b2ae2bef (127.15.196.253:39899), 977aabd1762642a79d0adb753b8d36f4 (127.15.196.252:36925)
I20260929 15:56:57.426419 6328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8c22b2ede2ad4e699d851ede8189d98b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "977aabd1762642a79d0adb753b8d36f4"
I20260929 15:56:57.426939 6328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:57.430507 6328 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c22b2ede2ad4e699d851ede8189d98b in term 1.
I20260929 15:56:57.431182 6152 leader_election.cc:302] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c22b2ede2ad4e699d851ede8189d98b, 977aabd1762642a79d0adb753b8d36f4; no voters:
I20260929 15:56:57.431710 6374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:56:57.440181 6374 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [term 1 LEADER]: Becoming Leader. State: Replica: 8c22b2ede2ad4e699d851ede8189d98b, State: Running, Role: LEADER
I20260929 15:56:57.440862 6374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } }
W20260929 15:56:57.467612 6156 tablet.cc:2369] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:56:57.479961 6384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c22b2ede2ad4e699d851ede8189d98b. Latest consensus state: current_term: 1 leader_uuid: "8c22b2ede2ad4e699d851ede8189d98b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } } }
I20260929 15:56:57.480495 6384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:57.484408 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.446081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41778 (local address 127.15.196.252:36925)
0929 15:56:57.446258 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.446264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.456826 (+ 10562us) server_negotiation.cc:407] Connection header received
0929 15:56:57.457012 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.457018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.457119 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.457226 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:57.460783 (+ 3557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.461948 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.465388 (+ 3440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.466147 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.466250 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.473407 (+ 7157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:57.473415 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:57.473420 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:57.473465 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:57.477583 (+ 4118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.479806 (+ 2223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.479812 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.479817 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.479989 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.480612 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.480619 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.480624 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.481012 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:57.481276 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:57.483662 (+ 2386us) server_negotiation.cc:299] Negotiation successful
0929 15:56:57.483979 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 15:56:57.487790 6278 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } attempt: 5
I20260929 15:56:57.493376 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.450951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50978 (local address 127.15.196.253:39899)
0929 15:56:57.451173 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.451181 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.457651 (+ 6470us) server_negotiation.cc:407] Connection header received
0929 15:56:57.458355 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.458363 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.458455 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.458546 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:57.461972 (+ 3426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.463110 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.470549 (+ 7439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.471440 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.471570 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.481662 (+ 10092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:57.481669 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:57.481675 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:57.481731 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:57.485698 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.490137 (+ 4439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.490147 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.490154 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.490398 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.490930 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.490938 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.490944 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.491437 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:57.491729 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:57.492473 (+ 744us) server_negotiation.cc:299] Negotiation successful
0929 15:56:57.492812 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 15:56:57.496528 6395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 29 15:56:57 dist-test-slave-c51v krb5kdc[6122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697416, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.253@KRBTEST.COM for kudu/127.15.196.252@KRBTEST.COM
I20260929 15:56:57.499414 6394 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 15:56:57.504899 6357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.444999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41740 (local address 127.15.196.254:35789)
0929 15:56:57.445178 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.445185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.453161 (+ 7976us) server_negotiation.cc:407] Connection header received
0929 15:56:57.453495 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.453505 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.453619 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.453712 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:57.463325 (+ 9613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.464690 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.472652 (+ 7962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.473505 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.473608 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.480797 (+ 7189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:57.480805 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:57.480810 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:57.480853 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:57.498117 (+ 17264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.502037 (+ 3920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.502044 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.502049 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.502228 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.502731 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.502740 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.502846 (+ 106us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.503313 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:57.503610 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:57.504208 (+ 598us) server_negotiation.cc:299] Negotiation successful
0929 15:56:57.504484 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 15:56:57.510149 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.489131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41786 (local address 127.15.196.252:36925)
0929 15:56:57.489317 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.489324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.489346 (+ 22us) server_negotiation.cc:407] Connection header received
0929 15:56:57.489402 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.489407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.489481 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.489584 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:57.490809 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.491975 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.493905 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.494656 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.494800 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.500452 (+ 5652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:56:57.500462 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:56:57.500469 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:56:57.500506 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:56:57.504631 (+ 4125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.506142 (+ 1511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.506150 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.506153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.506431 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:56:57.507179 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:56:57.507186 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:56:57.507189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:56:57.507863 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:56:57.508126 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:57.508990 (+ 864us) server_negotiation.cc:299] Negotiation successful
0929 15:56:57.509442 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 15:56:57.511780 6282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.488385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:36925 (local address 127.0.0.1:41786)
0929 15:56:57.488764 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0929 15:56:57.488786 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 15:56:57.488906 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:56:57.489809 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:56:57.489814 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:56:57.490358 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:56:57.490597 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:57.490614 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.492209 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.492216 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:57.493715 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:56:57.493732 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.495010 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.495017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:56:57.495333 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.496961 (+ 1628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:56:57.496986 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:56:57.500207 (+ 3221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:56:57.504898 (+ 4691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:57.504902 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:57.504906 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:57.505957 (+ 1051us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:57.506687 (+ 730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:56:57.506691 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:56:57.506694 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:56:57.506997 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:56:57.508283 (+ 1286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:56:57.508288 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:56:57.508542 (+ 254us) client_negotiation.cc:769] Sending connection context
0929 15:56:57.510712 (+ 2170us) client_negotiation.cc:240] Negotiation successful
0929 15:56:57.511181 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249}
I20260929 15:56:57.518121 6278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef: Bootstrap starting.
I20260929 15:56:57.524959 6278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef: Neither blocks nor log segments found. Creating new log.
I20260929 15:56:57.526959 6395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 15:56:57.527191 6278 log.cc:824] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef: Log is configured to *not* fsync() on all Append() calls
I20260929 15:56:57.511898 6135 master_runner.cc:186] Error getting master registration for 127.15.196.253:39899: OK, instance_id { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" instance_seqno: 1790697416757966 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 15:56:57.530781 6135 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 15:56:57.530710 6307 tablet.cc:2369] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:56:57.539868 6278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef: No bootstrap required, opened a new log
I20260929 15:56:57.543264 6328 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4 [term 1 FOLLOWER]: Refusing update from remote peer 8c22b2ede2ad4e699d851ede8189d98b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:56:57.544236 6384 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [LEADER]: Connected to new peer: Peer: permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:56:57.557469 6278 raft_consensus.cc:348] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } }
I20260929 15:56:57.558267 6278 raft_consensus.cc:729] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 870fe458763541cf99ca3a76b2ae2bef, State: Initialized, Role: FOLLOWER
I20260929 15:56:57.559163 6278 consensus_queue.cc:260] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } }
I20260929 15:56:57.570194 6252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8c22b2ede2ad4e699d851ede8189d98b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "870fe458763541cf99ca3a76b2ae2bef"
I20260929 15:56:57.570580 6251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:56:57.570927 6278 sys_catalog.cc:564] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:56:57.574555 6251 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef [term 1 FOLLOWER]: Refusing update from remote peer 8c22b2ede2ad4e699d851ede8189d98b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:56:57.575877 6400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } } }
I20260929 15:56:57.576376 6400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:56:57.578233 6252 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c22b2ede2ad4e699d851ede8189d98b in term 1.
I20260929 15:56:57.580444 6384 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [LEADER]: Connected to new peer: Peer: permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:56:57.597854 6400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c22b2ede2ad4e699d851ede8189d98b. Latest consensus state: current_term: 1 leader_uuid: "8c22b2ede2ad4e699d851ede8189d98b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } } }
I20260929 15:56:57.598254 6400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:56:57.601615 6360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c22b2ede2ad4e699d851ede8189d98b. Latest consensus state: current_term: 1 leader_uuid: "8c22b2ede2ad4e699d851ede8189d98b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } } }
I20260929 15:56:57.601985 6360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:56:57.602278 6398 mvcc.cc:205] Tried to move back new op lower bound from 7334696622247129088 to 7334696621845225472. Current Snapshot: MvccSnapshot[applied={T|T < 7334696622247129088}]
I20260929 15:56:57.603998 6399 mvcc.cc:205] Tried to move back new op lower bound from 7334696622247129088 to 7334696621845225472. Current Snapshot: MvccSnapshot[applied={T|T < 7334696622247129088}]
I20260929 15:56:57.607836 6374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c22b2ede2ad4e699d851ede8189d98b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } } }
I20260929 15:56:57.608170 6374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:57.608456 6374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c22b2ede2ad4e699d851ede8189d98b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } } }
I20260929 15:56:57.608728 6374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b [sys.catalog]: This master's current role is: LEADER
I20260929 15:56:57.610244 6395 catalog_manager.cc:1347] Generated new cluster ID: 99f5fd6e41c4410a8d812f8c44afe345
I20260929 15:56:57.610350 6395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:56:57.610934 6360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c22b2ede2ad4e699d851ede8189d98b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } } }
I20260929 15:56:57.611179 6360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:56:57.628731 6400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c22b2ede2ad4e699d851ede8189d98b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c22b2ede2ad4e699d851ede8189d98b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 35789 } } peers { permanent_uuid: "870fe458763541cf99ca3a76b2ae2bef" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 39899 } } peers { permanent_uuid: "977aabd1762642a79d0adb753b8d36f4" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 36925 } } }
I20260929 15:56:57.629025 6400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:56:57.645488 6395 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:56:57.648070 6395 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:56:57.662539 6395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c22b2ede2ad4e699d851ede8189d98b: Generated new TSK 0
I20260929 15:56:57.667618 6395 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:56:57.715306 6418 catalog_manager.cc:1259] Loaded cluster ID: 99f5fd6e41c4410a8d812f8c44afe345
I20260929 15:56:57.715533 6418 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef: loading cluster ID for follower catalog manager: success
I20260929 15:56:57.719533 6418 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef: acquiring CA information for follower catalog manager: success
I20260929 15:56:57.722337 6418 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 870fe458763541cf99ca3a76b2ae2bef: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 15:56:57.829041 6357 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:56:57.802828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41746 (local address 127.15.196.254:35789)
0929 15:56:57.803049 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.803055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.825759 (+ 22704us) server_negotiation.cc:407] Connection header received
0929 15:56:57.827484 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.827492 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.827580 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.827664 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:57.828788 (+ 1124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41746: BlockingRecv error: recv got EOF from 127.0.0.1:41746 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260929 15:56:57.831019 6356 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:56:57.809652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41788 (local address 127.15.196.252:36925)
0929 15:56:57.809835 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.809841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.827017 (+ 17176us) server_negotiation.cc:407] Connection header received
0929 15:56:57.829545 (+ 2528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.829551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.829629 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.829695 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:57.830765 (+ 1070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41788: BlockingRecv error: recv got EOF from 127.0.0.1:41788 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260929 15:56:57.833667 6283 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:56:57.823471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50988 (local address 127.15.196.253:39899)
0929 15:56:57.823705 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.823713 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.831607 (+ 7894us) server_negotiation.cc:407] Connection header received
0929 15:56:57.832146 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.832152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.832232 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.832314 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:57.833418 (+ 1104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50988: BlockingRecv error: recv got EOF from 127.0.0.1:50988 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 15:56:57.878775 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.862307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50990 (local address 127.15.196.253:39899)
0929 15:56:57.862507 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.862513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.870687 (+ 8174us) server_negotiation.cc:407] Connection header received
0929 15:56:57.871264 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.871271 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.871498 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.871629 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:57.872388 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.873561 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.875688 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.876372 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.876504 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.877225 (+ 721us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:57.877904 (+ 679us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:57.878064 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:57.878315 (+ 251us) server_negotiation.cc:299] Negotiation successful
0929 15:56:57.878413 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260929 15:56:57.926071 6356 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:56:57.913052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41796 (local address 127.15.196.252:36925)
0929 15:56:57.913273 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.913280 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.923884 (+ 10604us) server_negotiation.cc:407] Connection header received
0929 15:56:57.924465 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.924472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.924604 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.924690 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:57.925808 (+ 1118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41796: BlockingRecv error: recv got EOF from 127.0.0.1:41796 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 15:56:57.976049 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:57.961554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50992 (local address 127.15.196.253:39899)
0929 15:56:57.961780 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:57.961786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:57.968656 (+ 6870us) server_negotiation.cc:407] Connection header received
0929 15:56:57.969174 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:57.969183 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:57.969293 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:57.969368 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:57.970428 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.971651 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.973780 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:57.974434 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:57.974527 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:57.975152 (+ 625us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:57.975343 (+ 191us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:57.975407 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:57.975633 (+ 226us) server_negotiation.cc:299] Negotiation successful
0929 15:56:57.975719 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260929 15:56:58.019631 6356 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:56:58.010359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41800 (local address 127.15.196.252:36925)
0929 15:56:58.010664 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.010675 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.017777 (+ 7102us) server_negotiation.cc:407] Connection header received
0929 15:56:58.018029 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.018035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.018145 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.018245 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:58.019372 (+ 1127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41800: BlockingRecv error: recv got EOF from 127.0.0.1:41800 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260929 15:56:58.072388 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:58.057844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51006 (local address 127.15.196.253:39899)
0929 15:56:58.058048 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.058055 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.064999 (+ 6944us) server_negotiation.cc:407] Connection header received
0929 15:56:58.065501 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.065508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.065602 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.065668 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:58.066789 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.067949 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.070014 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.070730 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.070847 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:58.071491 (+ 644us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:58.071630 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:58.071691 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:58.071930 (+ 239us) server_negotiation.cc:299] Negotiation successful
0929 15:56:58.072032 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260929 15:56:58.082541 6356 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:56:58.080209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41806 (local address 127.15.196.252:36925)
0929 15:56:58.080397 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.080403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.080944 (+ 541us) server_negotiation.cc:407] Connection header received
0929 15:56:58.081212 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.081218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.081309 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.081379 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:58.082307 (+ 928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41806: BlockingRecv error: recv got EOF from 127.0.0.1:41806 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 15:56:58.102535 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:58.094985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51014 (local address 127.15.196.253:39899)
0929 15:56:58.095206 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.095216 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.095752 (+ 536us) server_negotiation.cc:407] Connection header received
0929 15:56:58.096011 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.096018 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.096141 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.096262 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:58.096729 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.098034 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.100177 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.101030 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.101179 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:58.101648 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:58.101799 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:58.101916 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:58.102085 (+ 169us) server_negotiation.cc:299] Negotiation successful
0929 15:56:58.102181 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
W20260929 15:56:58.115231 6356 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:56:58.112922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41814 (local address 127.15.196.252:36925)
0929 15:56:58.113162 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.113167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.113658 (+ 491us) server_negotiation.cc:407] Connection header received
0929 15:56:58.113845 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.113852 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.113963 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.114086 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:58.114970 (+ 884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41814: BlockingRecv error: recv got EOF from 127.0.0.1:41814 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
I20260929 15:56:58.134729 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:58.126967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51026 (local address 127.15.196.253:39899)
0929 15:56:58.127182 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.127190 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.127841 (+ 651us) server_negotiation.cc:407] Connection header received
0929 15:56:58.128050 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.128057 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.128175 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.128303 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:58.128975 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.130176 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.132314 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.133118 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.133259 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:58.133825 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:58.133984 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:58.134083 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:58.134269 (+ 186us) server_negotiation.cc:299] Negotiation successful
0929 15:56:58.134360 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260929 15:56:58.144820 6356 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:56:58.142312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41824 (local address 127.15.196.252:36925)
0929 15:56:58.142525 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.142533 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.143109 (+ 576us) server_negotiation.cc:407] Connection header received
0929 15:56:58.143346 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.143353 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.143456 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.143571 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:58.144547 (+ 976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41824: BlockingRecv error: recv got EOF from 127.0.0.1:41824 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 15:56:58.172232 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:58.165187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51032 (local address 127.15.196.253:39899)
0929 15:56:58.165414 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.165421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.165918 (+ 497us) server_negotiation.cc:407] Connection header received
0929 15:56:58.166080 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.166086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.166176 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.166289 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:58.166719 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.167994 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.169927 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.170595 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.170734 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:58.171276 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:58.171467 (+ 191us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:58.171581 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:58.171780 (+ 199us) server_negotiation.cc:299] Negotiation successful
0929 15:56:58.171873 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
W20260929 15:56:58.187551 6356 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:56:58.185371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41836 (local address 127.15.196.252:36925)
0929 15:56:58.185579 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.185586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.185990 (+ 404us) server_negotiation.cc:407] Connection header received
0929 15:56:58.186192 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.186197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.186311 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.186456 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:58.187235 (+ 779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41836: BlockingRecv error: recv got EOF from 127.0.0.1:41836 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260929 15:56:58.229497 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:58.223073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51038 (local address 127.15.196.253:39899)
0929 15:56:58.223393 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.223404 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.223769 (+ 365us) server_negotiation.cc:407] Connection header received
0929 15:56:58.223946 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.223953 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.224067 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.224196 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:58.224608 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.225892 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.227205 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.228024 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.228201 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:58.228527 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:58.228666 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:58.228789 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:58.229008 (+ 219us) server_negotiation.cc:299] Negotiation successful
0929 15:56:58.229095 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260929 15:56:58.240108 6356 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:56:58.237416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41850 (local address 127.15.196.252:36925)
0929 15:56:58.237609 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.237615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.238155 (+ 540us) server_negotiation.cc:407] Connection header received
0929 15:56:58.238318 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.238326 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.238438 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.239075 (+ 637us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:58.239879 (+ 804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41850: BlockingRecv error: recv got EOF from 127.0.0.1:41850 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 15:56:58.316781 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:58.308642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51050 (local address 127.15.196.253:39899)
0929 15:56:58.308848 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.308855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.311730 (+ 2875us) server_negotiation.cc:407] Connection header received
0929 15:56:58.311999 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.312005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.312083 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.312197 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:58.312736 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.313512 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.314922 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.315435 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.315587 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:58.316013 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:58.316122 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:58.316233 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:58.316392 (+ 159us) server_negotiation.cc:299] Negotiation successful
0929 15:56:58.316479 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260929 15:56:58.324661 6356 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:56:58.322777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41852 (local address 127.15.196.252:36925)
0929 15:56:58.323024 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.323031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.323407 (+ 376us) server_negotiation.cc:407] Connection header received
0929 15:56:58.323594 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.323598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.323676 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.323759 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:56:58.324502 (+ 743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41852: BlockingRecv error: recv got EOF from 127.0.0.1:41852 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 15:56:58.404731 6371 catalog_manager.cc:1259] Loaded cluster ID: 99f5fd6e41c4410a8d812f8c44afe345
I20260929 15:56:58.404861 6371 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4: loading cluster ID for follower catalog manager: success
I20260929 15:56:58.407467 6371 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4: acquiring CA information for follower catalog manager: success
I20260929 15:56:58.409264 6371 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 977aabd1762642a79d0adb753b8d36f4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 15:56:58.461340 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:58.456246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51060 (local address 127.15.196.253:39899)
0929 15:56:58.456534 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.456541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.456821 (+ 280us) server_negotiation.cc:407] Connection header received
0929 15:56:58.457062 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.457066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.457134 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.457235 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:58.457630 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.458313 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.459673 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.460072 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.460208 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:58.460627 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:58.460737 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:58.460840 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:58.460995 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 15:56:58.461074 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260929 15:56:58.472985 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:58.467236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41864 (local address 127.15.196.252:36925)
0929 15:56:58.467484 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.467489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.467799 (+ 310us) server_negotiation.cc:407] Connection header received
0929 15:56:58.467967 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.467971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.468118 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.468214 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:58.468596 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.469260 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.470649 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.471036 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.471170 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:58.471778 (+ 608us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:58.472348 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:58.472488 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:58.472600 (+ 112us) server_negotiation.cc:299] Negotiation successful
0929 15:56:58.472672 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260929 15:56:58.493897 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:58.479232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41762 (local address 127.15.196.254:35789)
0929 15:56:58.486911 (+ 7679us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.486920 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.486955 (+ 35us) server_negotiation.cc:407] Connection header received
0929 15:56:58.487087 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.487095 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.487364 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.487546 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:58.488169 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.489158 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.491084 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.491734 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.491876 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:58.492432 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:58.492772 (+ 340us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:58.492904 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:58.493454 (+ 550us) server_negotiation.cc:299] Negotiation successful
0929 15:56:58.493535 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7545,"thread_start_us":7456,"threads_started":1}
I20260929 15:56:58.493927 6283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:58.479577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51076 (local address 127.15.196.253:39899)
0929 15:56:58.479806 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.479814 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.487803 (+ 7989us) server_negotiation.cc:407] Connection header received
0929 15:56:58.488017 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.488023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.488109 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.488206 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:58.488815 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.489881 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.491547 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.492236 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.492408 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:58.493125 (+ 717us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:58.493261 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:58.493361 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:58.493553 (+ 192us) server_negotiation.cc:299] Negotiation successful
0929 15:56:58.493641 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 15:56:58.494092 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:56:58.479376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.15.196.252:36925)
0929 15:56:58.479621 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0929 15:56:58.479630 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:56:58.487358 (+ 7728us) server_negotiation.cc:407] Connection header received
0929 15:56:58.487669 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:56:58.487676 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:56:58.487790 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:56:58.487854 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:56:58.488677 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.489769 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.491915 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:56:58.492625 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:56:58.492719 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:56:58.493372 (+ 653us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:56:58.493507 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:56:58.493593 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 15:56:58.493745 (+ 152us) server_negotiation.cc:299] Negotiation successful
0929 15:56:58.493823 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260929 15:56:58.498889 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 6135
I20260929 15:56:58.500255 6198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:58.611440 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 6135
W20260929 15:56:58.632387 6303 connection.cc:535] server connection from 127.0.0.1:41768 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 15:56:58.632781 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 6210
I20260929 15:56:58.633855 6273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:58.741308 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 6210
I20260929 15:56:58.761826 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 6286
I20260929 15:56:58.762981 6348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:56:58.871007 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 6286
2026-09-29T15:56:58Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4471 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 15:56:58 dist-test-slave-c51v krb5kdc[6532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 15:56:58 dist-test-slave-c51v krb5kdc[6532](info): set up 2 sockets
Sep 29 15:56:58 dist-test-slave-c51v krb5kdc[6532](info): commencing operation
krb5kdc: starting...
W20260929 15:56:59.990065 16147 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/test-user.keytab.
Sep 29 15:57:00 dist-test-slave-c51v krb5kdc[6532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T15:57:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T15:57:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.196.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
Entry for principal kudu/127.15.196.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab.
I20260929 15:57:00.244593 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:38211
--webserver_interface=127.15.196.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:40807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:38211,127.15.196.253:36989,127.15.196.252:35449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 15:57:00.575681 6545 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 15:57:00.576051 6545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:57:00.576172 6545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:57:00.576293 6545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:57:00.583642 6545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:57:00.583737 6545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:57:00.583844 6545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:57:00.583907 6545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:57:00.583994 6545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:57:00.593910 6545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:40807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:38211,127.15.196.253:36989,127.15.196.252:35449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.254.keytab
--principal=kudu/127.15.196.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.254:38211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.196.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:57:00.595232 6545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:57:00.596884 6545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:57:00.608325 6553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:57:00.608362 6551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:57:00.608386 6550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:57:00.609539 6545 server_base.cc:1034] running on GCE node
I20260929 15:57:00.610350 6545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:57:00.611779 6545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:57:00.613014 6545 hybrid_clock.cc:648] HybridClock initialized: now 1790697420612950 us; error 40 us; skew 500 ppm
I20260929 15:57:00.613337 6545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:57:00 dist-test-slave-c51v krb5kdc[6532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697420, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:57:00.616369 6545 init.cc:377] Logged in from keytab as kudu/127.15.196.254@KRBTEST.COM (short username kudu)
I20260929 15:57:00.617722 6545 webserver.cc:460] Webserver started at http://127.15.196.254:41135/ using document root <none> and password file <none>
I20260929 15:57:00.618425 6545 fs_manager.cc:359] Metadata directory not provided
I20260929 15:57:00.618516 6545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:57:00.618778 6545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:57:00.621662 6545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/data/instance:
uuid: "0705c9f8915749f78b913da2ab21ad6b"
format_stamp: "Formatted at 2026-09-29 15:57:00 on dist-test-slave-c51v"
server_key: "aa848cdbec439adafdba42af4e3731d4"
server_key_iv: "19ba93f432f40872b6d91e4fc999e68e"
server_key_version: "encryptionkey@0"
I20260929 15:57:00.622378 6545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/wal/instance:
uuid: "0705c9f8915749f78b913da2ab21ad6b"
format_stamp: "Formatted at 2026-09-29 15:57:00 on dist-test-slave-c51v"
server_key: "aa848cdbec439adafdba42af4e3731d4"
server_key_iv: "19ba93f432f40872b6d91e4fc999e68e"
server_key_version: "encryptionkey@0"
I20260929 15:57:00.627346 6545 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 15:57:00.630652 6560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:57:00.632009 6545 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 15:57:00.632157 6545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/wal
uuid: "0705c9f8915749f78b913da2ab21ad6b"
format_stamp: "Formatted at 2026-09-29 15:57:00 on dist-test-slave-c51v"
server_key: "aa848cdbec439adafdba42af4e3731d4"
server_key_iv: "19ba93f432f40872b6d91e4fc999e68e"
server_key_version: "encryptionkey@0"
I20260929 15:57:00.632329 6545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:57:00.651665 6545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:57:00.655463 6545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:57:00.655679 6545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:57:00.674207 6545 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.254:38211
I20260929 15:57:00.674216 6612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.254:38211 every 8 connection(s)
I20260929 15:57:00.675652 6545 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/data/info.pb
I20260929 15:57:00.679114 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 6545
I20260929 15:57:00.679355 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-0/wal/instance
I20260929 15:57:00.679715 6613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:57:00.679947 16147 external_mini_cluster.cc:1428] Setting key 80aea6f1c669b0f0d7906885641d1bfe
I20260929 15:57:00.689785 6613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:57:00 dist-test-slave-c51v krb5kdc[6532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697420, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.196.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.253@KRBTEST.COM" created.
I20260929 15:57:00.716076 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:00.693536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38211 (local address 127.0.0.1:50144)
0929 15:57:00.694471 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:00.694530 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 15:57:00.697367 (+ 2837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:57:00.698292 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:57:00.698302 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:57:00.699134 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:57:00.699711 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:00.699739 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:00.701299 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:00.701307 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:00.703235 (+ 1928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:00.703253 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:00.704196 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:00.704205 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:00.704435 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:00.705233 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:57:00.705265 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:57:00.708216 (+ 2951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:57:00.711745 (+ 3529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:00.711758 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:00.711770 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:00.712275 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:00.712670 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:00.712676 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:00.712680 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:00.712837 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:00.713581 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:57:00.713589 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:57:00.713809 (+ 220us) client_negotiation.cc:769] Sending connection context
0929 15:57:00.714045 (+ 236us) client_negotiation.cc:240] Negotiation successful
0929 15:57:00.714658 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":168,"thread_start_us":375,"threads_started":1}
I20260929 15:57:00.716106 6617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:00.693536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50144 (local address 127.15.196.254:38211)
0929 15:57:00.695470 (+ 1934us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:00.695476 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:00.695527 (+ 51us) server_negotiation.cc:407] Connection header received
0929 15:57:00.697782 (+ 2255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:00.697788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:00.698043 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:00.698211 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:00.699940 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:00.701143 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:00.703391 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:00.703946 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:00.704078 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:00.708510 (+ 4432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:57:00.708524 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:57:00.708544 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:57:00.708583 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:57:00.711618 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:00.712416 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:00.712422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:00.712424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:00.712565 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:00.712953 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:00.712957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:00.712959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:00.713439 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:57:00.713646 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:00.714166 (+ 520us) server_negotiation.cc:299] Negotiation successful
0929 15:57:00.714550 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":1747,"spinlock_wait_cycles":23424,"thread_start_us":1491,"threads_started":1}
I20260929 15:57:00.719208 6613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } has no permanent_uuid. Determining permanent_uuid...
I20260929 15:57:00.720532 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:00.720024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:36989 (local address 127.0.0.1:42386)
0929 15:57:00.720234 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:00.720364 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260929 15:57:00.720852 6564 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.196.253:36989: connect: Connection refused (error 111)
I20260929 15:57:00.722309 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:00.721997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:36989 (local address 127.0.0.1:42390)
0929 15:57:00.722120 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:00.722185 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 15:57:00.722518 6613 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.253:36989: Network error: Client connection negotiation failed: client connection to 127.15.196.253:36989: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.253.keytab.
Entry for principal kudu/127.15.196.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.253.keytab.
I20260929 15:57:00.742741 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.253:36989
--webserver_interface=127.15.196.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.253.keytab
--principal=kudu/127.15.196.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:40807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:38211,127.15.196.253:36989,127.15.196.252:35449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:57:00.771839 6613 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } attempt: 1
I20260929 15:57:00.773095 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:00.772543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:36989 (local address 127.0.0.1:42392)
0929 15:57:00.772773 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:00.772928 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260929 15:57:00.774178 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:00.773781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:36989 (local address 127.0.0.1:42408)
0929 15:57:00.773930 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:00.774018 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 15:57:00.774380 6613 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.253:36989: Network error: Client connection negotiation failed: client connection to 127.15.196.253:36989: connect: Connection refused (error 111)
I20260929 15:57:00.842624 6613 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } attempt: 2
I20260929 15:57:00.843891 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:00.843330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:36989 (local address 127.0.0.1:42410)
0929 15:57:00.843605 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:00.843718 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 15:57:00.844892 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:00.844511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:36989 (local address 127.0.0.1:42418)
0929 15:57:00.844665 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:00.844748 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260929 15:57:00.845080 6613 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.253:36989: Network error: Client connection negotiation failed: client connection to 127.15.196.253:36989: connect: Connection refused (error 111)
I20260929 15:57:00.936360 6613 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } attempt: 3
I20260929 15:57:00.937693 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:00.937089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:36989 (local address 127.0.0.1:42426)
0929 15:57:00.937342 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:00.937485 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260929 15:57:00.938956 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:00.938465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:36989 (local address 127.0.0.1:42428)
0929 15:57:00.938643 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:00.938756 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260929 15:57:00.939150 6613 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.253:36989: Network error: Client connection negotiation failed: client connection to 127.15.196.253:36989: connect: Connection refused (error 111)
W20260929 15:57:01.075896 6620 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 15:57:01.076249 6620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:57:01.076367 6620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:57:01.076499 6620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260929 15:57:01.082437 6613 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } attempt: 4
I20260929 15:57:01.083689 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.083066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:36989 (local address 127.0.0.1:42432)
0929 15:57:01.083406 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.083514 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260929 15:57:01.084200 6620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:57:01.084316 6620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:57:01.084429 6620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:57:01.084506 6620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:57:01.084597 6620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:57:01.084782 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.084374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:36989 (local address 127.0.0.1:42444)
0929 15:57:01.084540 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.084646 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.253:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 15:57:01.085009 6613 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.253:36989: Network error: Client connection negotiation failed: client connection to 127.15.196.253:36989: connect: Connection refused (error 111)
I20260929 15:57:01.094565 6620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:40807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:38211,127.15.196.253:36989,127.15.196.252:35449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.253.keytab
--principal=kudu/127.15.196.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.253:36989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.196.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:57:01.095911 6620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:57:01.097158 6620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 15:57:01.108597 6626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:57:01.108654 6628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:57:01.108633 6625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:57:01.109411 6620 server_base.cc:1034] running on GCE node
I20260929 15:57:01.110126 6620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:57:01.111446 6620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:57:01.112668 6620 hybrid_clock.cc:648] HybridClock initialized: now 1790697421112600 us; error 42 us; skew 500 ppm
I20260929 15:57:01.112951 6620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:57:01 dist-test-slave-c51v krb5kdc[6532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697421, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:57:01.116070 6620 init.cc:377] Logged in from keytab as kudu/127.15.196.253@KRBTEST.COM (short username kudu)
I20260929 15:57:01.117668 6620 webserver.cc:460] Webserver started at http://127.15.196.253:37523/ using document root <none> and password file <none>
I20260929 15:57:01.118369 6620 fs_manager.cc:359] Metadata directory not provided
I20260929 15:57:01.118479 6620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:57:01.118816 6620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:57:01.121749 6620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/data/instance:
uuid: "aaaa24c395344612a07fd9593cae4d56"
format_stamp: "Formatted at 2026-09-29 15:57:01 on dist-test-slave-c51v"
server_key: "590a553b1e91ae9cca1cded68a7173bb"
server_key_iv: "0acd05e6f0fd934dea4e041a391a29fc"
server_key_version: "encryptionkey@0"
I20260929 15:57:01.122541 6620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/wal/instance:
uuid: "aaaa24c395344612a07fd9593cae4d56"
format_stamp: "Formatted at 2026-09-29 15:57:01 on dist-test-slave-c51v"
server_key: "590a553b1e91ae9cca1cded68a7173bb"
server_key_iv: "0acd05e6f0fd934dea4e041a391a29fc"
server_key_version: "encryptionkey@0"
I20260929 15:57:01.127737 6620 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 15:57:01.131795 6635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:57:01.133173 6620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:57:01.133352 6620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/wal
uuid: "aaaa24c395344612a07fd9593cae4d56"
format_stamp: "Formatted at 2026-09-29 15:57:01 on dist-test-slave-c51v"
server_key: "590a553b1e91ae9cca1cded68a7173bb"
server_key_iv: "0acd05e6f0fd934dea4e041a391a29fc"
server_key_version: "encryptionkey@0"
I20260929 15:57:01.133577 6620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:57:01.147811 6620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:57:01.151755 6620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:57:01.152019 6620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:57:01.170856 6620 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.253:36989
I20260929 15:57:01.170876 6686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.253:36989 every 8 connection(s)
I20260929 15:57:01.172266 6620 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/data/info.pb
I20260929 15:57:01.175944 6687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:57:01.177824 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 6620
I20260929 15:57:01.178010 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-1/wal/instance
I20260929 15:57:01.178473 16147 external_mini_cluster.cc:1428] Setting key 73207f1134bb84b6e036f4fca05b5991
I20260929 15:57:01.182561 6687 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.196.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.196.252@KRBTEST.COM" created.
Sep 29 15:57:01 dist-test-slave-c51v krb5kdc[6532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697421, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.253@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:57:01.215197 6617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.185145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50154 (local address 127.15.196.254:38211)
0929 15:57:01.185381 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:01.185390 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:01.186570 (+ 1180us) server_negotiation.cc:407] Connection header received
0929 15:57:01.189616 (+ 3046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:01.189620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:01.189690 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:01.189776 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:01.191897 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.192588 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.195068 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.195660 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.195807 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.208078 (+ 12271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:57:01.208084 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:57:01.208087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:57:01.208127 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:57:01.211274 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.212316 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.212324 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.212330 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.212500 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.212997 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.213006 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.213012 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.213473 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:57:01.213647 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:01.214519 (+ 872us) server_negotiation.cc:299] Negotiation successful
0929 15:57:01.214786 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 15:57:01.216194 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.185556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38211 (local address 127.0.0.1:50154)
0929 15:57:01.186379 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.186439 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 15:57:01.189301 (+ 2862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:57:01.190056 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:57:01.190070 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:57:01.190851 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:57:01.191683 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:01.191711 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.192754 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.192762 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:01.194870 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:01.194887 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.195915 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.195924 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:01.196274 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.202439 (+ 6165us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:57:01.202489 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:57:01.207789 (+ 5300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:57:01.211484 (+ 3695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:01.211494 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:01.211514 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:01.212151 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:01.212640 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:01.212646 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:01.212652 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:01.212869 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:01.213606 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:57:01.213616 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:57:01.214086 (+ 470us) client_negotiation.cc:769] Sending connection context
0929 15:57:01.214363 (+ 277us) client_negotiation.cc:240] Negotiation successful
0929 15:57:01.214891 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":317,"threads_started":1}
I20260929 15:57:01.218031 6687 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:57:01 dist-test-slave-c51v krb5kdc[6532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697421, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.253@KRBTEST.COM for kudu/127.15.196.253@KRBTEST.COM
I20260929 15:57:01.235152 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.218859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:36989 (local address 127.0.0.1:42452)
0929 15:57:01.219049 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.219067 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:57:01.219178 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:57:01.220277 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:57:01.220280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:57:01.220630 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:57:01.220782 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:01.220789 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.222799 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.222803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:01.223676 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:01.223686 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.224944 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.224948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:01.225064 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.225636 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:57:01.225653 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:57:01.227830 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:57:01.232847 (+ 5017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:01.232851 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:01.232854 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:01.233216 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:01.233633 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:01.233637 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:01.233639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:01.233766 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:01.234486 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:57:01.234493 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:57:01.234593 (+ 100us) client_negotiation.cc:769] Sending connection context
0929 15:57:01.234696 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 15:57:01.234888 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260929 15:57:01.235556 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.219032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42452 (local address 127.15.196.253:36989)
0929 15:57:01.219773 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:01.219782 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:01.219814 (+ 32us) server_negotiation.cc:407] Connection header received
0929 15:57:01.219868 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:01.219877 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:01.220124 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:01.220271 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:01.220973 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.222666 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.223803 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.224787 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.224948 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.228079 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:57:01.228098 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:57:01.228120 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:57:01.228169 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:57:01.232684 (+ 4515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.233355 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.233365 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.233370 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.233521 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.233879 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.233885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.233888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.234351 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:57:01.234574 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:01.234869 (+ 295us) server_negotiation.cc:299] Negotiation successful
0929 15:57:01.235054 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":393,"threads_started":1}
I20260929 15:57:01.238204 6687 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } has no permanent_uuid. Determining permanent_uuid...
I20260929 15:57:01.239445 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.238856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39162)
0929 15:57:01.239079 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.239240 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260929 15:57:01.239707 6637 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
I20260929 15:57:01.241144 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.240848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39164)
0929 15:57:01.240984 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.241055 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 15:57:01.241346 6687 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:35449: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.196.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.252.keytab.
Entry for principal kudu/127.15.196.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.252.keytab.
I20260929 15:57:01.243522 16147 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6fGZba/build/asan/bin/kudu
/tmp/dist-test-task6fGZba/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.252:35449
--webserver_interface=127.15.196.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.252.keytab
--principal=kudu/127.15.196.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.196.212:40807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.196.254:38211,127.15.196.253:36989,127.15.196.252:35449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 15:57:01.290689 6687 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } attempt: 1
I20260929 15:57:01.291990 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.291496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39174)
0929 15:57:01.291702 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.291838 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260929 15:57:01.293037 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.292689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39178)
0929 15:57:01.292848 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.292932 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 15:57:01.293265 6687 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:35449: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
I20260929 15:57:01.361516 6687 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } attempt: 2
I20260929 15:57:01.362731 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.362250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39180)
0929 15:57:01.362482 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.362574 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260929 15:57:01.363849 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.363512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39188)
0929 15:57:01.363668 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.363735 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260929 15:57:01.364108 6687 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:35449: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
I20260929 15:57:01.384279 6613 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } attempt: 5
Sep 29 15:57:01 dist-test-slave-c51v krb5kdc[6532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697420, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for kudu/127.15.196.253@KRBTEST.COM
I20260929 15:57:01.397720 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.385150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42464 (local address 127.15.196.253:36989)
0929 15:57:01.385353 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:01.385359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:01.385383 (+ 24us) server_negotiation.cc:407] Connection header received
0929 15:57:01.385495 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:01.385500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:01.385586 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:01.385687 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:01.386475 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.387171 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.388454 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.388899 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.389027 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.392485 (+ 3458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:57:01.392490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:57:01.392494 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:57:01.392527 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:57:01.395163 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.396014 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.396018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.396020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.396132 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.396527 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.396530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.396532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.396793 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:57:01.396901 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:01.397321 (+ 420us) server_negotiation.cc:299] Negotiation successful
0929 15:57:01.397505 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260929 15:57:01.397859 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.385040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:36989 (local address 127.0.0.1:42464)
0929 15:57:01.385226 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.385249 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 15:57:01.385380 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:57:01.385725 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:57:01.385731 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:57:01.386143 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:57:01.386311 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:01.386320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.387357 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.387360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:01.388297 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:01.388309 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.389055 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.389060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:01.389277 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.389926 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:57:01.389943 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:57:01.392291 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:57:01.395348 (+ 3057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:01.395353 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:01.395357 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:01.395904 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:01.396234 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:01.396239 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:01.396243 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:01.396423 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:01.396903 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:57:01.396908 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:57:01.397082 (+ 174us) client_negotiation.cc:769] Sending connection context
0929 15:57:01.397232 (+ 150us) client_negotiation.cc:240] Negotiation successful
0929 15:57:01.397524 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 15:57:01.398980 6613 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } has no permanent_uuid. Determining permanent_uuid...
I20260929 15:57:01.400243 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.399813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39196)
0929 15:57:01.399994 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.400089 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 15:57:01.401292 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.400827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39208)
0929 15:57:01.401011 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.401123 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260929 15:57:01.401528 6613 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:35449: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
I20260929 15:57:01.452826 6613 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } attempt: 1
I20260929 15:57:01.454159 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.453614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39212)
0929 15:57:01.453844 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.453960 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260929 15:57:01.455241 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.454788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39226)
0929 15:57:01.454983 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.455086 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260929 15:57:01.455363 6687 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } attempt: 3
W20260929 15:57:01.455585 6613 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:35449: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
I20260929 15:57:01.456378 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.455997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39228)
0929 15:57:01.456165 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.456248 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 15:57:01.457460 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.457071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39242)
0929 15:57:01.457291 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.457362 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
W20260929 15:57:01.457701 6687 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:35449: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
I20260929 15:57:01.523792 6613 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } attempt: 2
I20260929 15:57:01.524986 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.524513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39248)
0929 15:57:01.524732 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.524828 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260929 15:57:01.525995 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.525702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39260)
0929 15:57:01.525822 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.525886 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 15:57:01.526206 6613 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:35449: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
W20260929 15:57:01.580360 6694 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 15:57:01.580682 6694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 15:57:01.580749 6694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 15:57:01.580830 6694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 15:57:01.588061 6694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 15:57:01.588150 6694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 15:57:01.588204 6694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 15:57:01.588253 6694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 15:57:01.588310 6694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 15:57:01.598366 6694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.196.212:40807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.196.254:38211,127.15.196.253:36989,127.15.196.252:35449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/krb5kdc/kudu_127.15.196.252.keytab
--principal=kudu/127.15.196.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.196.252:35449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.196.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 13:57:48 UTC on ccycloud.quasar-bocjyr.root.comops.site
build id 4680287
ASAN enabled
I20260929 15:57:01.599726 6694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 15:57:01.600908 6687 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } attempt: 4
I20260929 15:57:01.601087 6694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 15:57:01.602078 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.601608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39270)
0929 15:57:01.601825 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.601912 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260929 15:57:01.602974 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.602668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39282)
0929 15:57:01.602812 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.602875 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 15:57:01.603153 6687 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:35449: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
W20260929 15:57:01.613260 6699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:57:01.613358 6700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 15:57:01.613384 6702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 15:57:01.614279 6694 server_base.cc:1034] running on GCE node
I20260929 15:57:01.615111 6694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 15:57:01.616688 6694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 15:57:01.617911 6694 hybrid_clock.cc:648] HybridClock initialized: now 1790697421617846 us; error 34 us; skew 500 ppm
I20260929 15:57:01.618257 6694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 15:57:01 dist-test-slave-c51v krb5kdc[6532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697421, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 15:57:01.621371 6694 init.cc:377] Logged in from keytab as kudu/127.15.196.252@KRBTEST.COM (short username kudu)
I20260929 15:57:01.622785 6694 webserver.cc:460] Webserver started at http://127.15.196.252:33229/ using document root <none> and password file <none>
I20260929 15:57:01.623565 6694 fs_manager.cc:359] Metadata directory not provided
I20260929 15:57:01.623679 6694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 15:57:01.624008 6694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 15:57:01.626880 6694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/data/instance:
uuid: "711d2b55eb2b4d829bfb678218cd7f31"
format_stamp: "Formatted at 2026-09-29 15:57:01 on dist-test-slave-c51v"
server_key: "a0c4f4c7c600dfe59047f63abd91a70b"
server_key_iv: "aa9e12f266dde426543c00ab26a88d3e"
server_key_version: "encryptionkey@0"
I20260929 15:57:01.627699 6694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/wal/instance:
uuid: "711d2b55eb2b4d829bfb678218cd7f31"
format_stamp: "Formatted at 2026-09-29 15:57:01 on dist-test-slave-c51v"
server_key: "a0c4f4c7c600dfe59047f63abd91a70b"
server_key_iv: "aa9e12f266dde426543c00ab26a88d3e"
server_key_version: "encryptionkey@0"
I20260929 15:57:01.632417 6613 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } attempt: 3
I20260929 15:57:01.632683 6694 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260929 15:57:01.633443 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.632997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39298)
0929 15:57:01.633219 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.633299 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260929 15:57:01.634450 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.634118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39300)
0929 15:57:01.634293 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.634360 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260929 15:57:01.634629 6613 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.196.252:35449: Network error: Client connection negotiation failed: client connection to 127.15.196.252:35449: connect: Connection refused (error 111)
I20260929 15:57:01.636292 6710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 15:57:01.637701 6694 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 15:57:01.637868 6694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/data,/tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/wal
uuid: "711d2b55eb2b4d829bfb678218cd7f31"
format_stamp: "Formatted at 2026-09-29 15:57:01 on dist-test-slave-c51v"
server_key: "a0c4f4c7c600dfe59047f63abd91a70b"
server_key_iv: "aa9e12f266dde426543c00ab26a88d3e"
server_key_version: "encryptionkey@0"
I20260929 15:57:01.638079 6694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 15:57:01.659777 6694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 15:57:01.663591 6694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 15:57:01.663864 6694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 15:57:01.682442 6694 rpc_server.cc:307] RPC server started. Bound to: 127.15.196.252:35449
I20260929 15:57:01.682466 6761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.196.252:35449 every 8 connection(s)
I20260929 15:57:01.683802 6694 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/data/info.pb
I20260929 15:57:01.687381 6762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 15:57:01.688395 16147 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6fGZba/build/asan/bin/kudu as pid 6694
I20260929 15:57:01.688601 16147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6fGZba/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697059131395-16147-0/minicluster-data/master-2/wal/instance
I20260929 15:57:01.689037 16147 external_mini_cluster.cc:1428] Setting key 8aeedeedec2af5cfba6ddc1097bb8d21
I20260929 15:57:01.696945 6762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:57:01 dist-test-slave-c51v krb5kdc[6532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.252@KRBTEST.COM
Sep 29 15:57:01 dist-test-slave-c51v krb5kdc[6532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697421, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.252@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:57:01.713155 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.690907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39314 (local address 127.15.196.252:35449)
0929 15:57:01.691689 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:01.691705 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:01.691764 (+ 59us) server_negotiation.cc:407] Connection header received
0929 15:57:01.693885 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:01.693912 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:01.694445 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:01.694888 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:01.696223 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.698186 (+ 1963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.699591 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.700642 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.700873 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.705882 (+ 5009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:57:01.705892 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:57:01.705906 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:57:01.705951 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:57:01.709338 (+ 3387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.710005 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.710017 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.710023 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.710167 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.710543 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.710548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.710550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.711026 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:57:01.711370 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:01.711761 (+ 391us) server_negotiation.cc:299] Negotiation successful
0929 15:57:01.712167 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":346,"threads_started":1}
I20260929 15:57:01.714941 6617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.699746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50168 (local address 127.15.196.254:38211)
0929 15:57:01.699969 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:01.699976 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:01.700732 (+ 756us) server_negotiation.cc:407] Connection header received
0929 15:57:01.700903 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:01.700909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:01.700991 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:01.701105 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:01.702143 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.703333 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.704933 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.705619 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.705745 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.709623 (+ 3878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:57:01.709628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:57:01.709631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:57:01.709663 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:57:01.712380 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.713065 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.713075 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.713077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.713228 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.713612 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.713620 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.713624 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.713874 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:57:01.714039 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:01.714532 (+ 493us) server_negotiation.cc:299] Negotiation successful
0929 15:57:01.714696 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 15:57:01.715150 6766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.699755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.254:38211 (local address 127.0.0.1:50168)
0929 15:57:01.700577 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.700627 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 15:57:01.700781 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:57:01.701156 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:57:01.701161 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:57:01.701740 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:57:01.701994 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:01.702002 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.703471 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.703474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:01.704806 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:01.704817 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.705881 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.705889 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:01.706114 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.706785 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:57:01.706808 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:57:01.709436 (+ 2628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:57:01.712507 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:01.712519 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:01.712526 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:01.712932 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:01.713378 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:01.713382 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:01.713385 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:01.713507 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:01.714018 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:57:01.714025 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:57:01.714207 (+ 182us) client_negotiation.cc:769] Sending connection context
0929 15:57:01.714466 (+ 259us) client_negotiation.cc:240] Negotiation successful
0929 15:57:01.714754 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":408,"threads_started":1}
I20260929 15:57:01.716530 6762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:57:01 dist-test-slave-c51v krb5kdc[6532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697421, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.252@KRBTEST.COM for kudu/127.15.196.253@KRBTEST.COM
Sep 29 15:57:01 dist-test-slave-c51v krb5kdc[6532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.254@KRBTEST.COM
I20260929 15:57:01.729043 6766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.717272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.253:36989 (local address 127.0.0.1:42468)
0929 15:57:01.717414 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.717432 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 15:57:01.717527 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:57:01.717878 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:57:01.717882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:57:01.718228 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:57:01.718350 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:01.718357 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.719316 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.719324 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:01.720175 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:01.720185 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.720866 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.720871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:01.721089 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.721584 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:57:01.721600 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:57:01.723726 (+ 2126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:57:01.726686 (+ 2960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:01.726692 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:01.726697 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:01.727037 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:01.727476 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:01.727479 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:01.727482 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:01.727609 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:01.728125 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:57:01.728130 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:57:01.728276 (+ 146us) client_negotiation.cc:769] Sending connection context
0929 15:57:01.728411 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 15:57:01.728686 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 15:57:01.729084 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.717442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42468 (local address 127.15.196.253:36989)
0929 15:57:01.717626 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:01.717632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:01.717648 (+ 16us) server_negotiation.cc:407] Connection header received
0929 15:57:01.717703 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:01.717707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:01.717760 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:01.717859 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:01.718484 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.719163 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.720309 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.720726 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.720845 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.723913 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:57:01.723920 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:57:01.723924 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:57:01.723971 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:57:01.726564 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.727155 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.727165 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.727168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.727352 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.727742 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.727747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.727750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.728004 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:57:01.728161 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:01.728500 (+ 339us) server_negotiation.cc:299] Negotiation successful
0929 15:57:01.728709 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 15:57:01.730094 6617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.716245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50170 (local address 127.15.196.254:38211)
0929 15:57:01.716407 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:01.716411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:01.716440 (+ 29us) server_negotiation.cc:407] Connection header received
0929 15:57:01.716604 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:01.716607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:01.716674 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:01.716771 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:01.718069 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.718849 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.720104 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.720512 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.720639 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.724720 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:57:01.724725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:57:01.724728 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:57:01.724757 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:57:01.727357 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.727979 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.727985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.727989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.728124 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.728474 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.728479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.728483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.728742 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:57:01.729028 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:01.729349 (+ 321us) server_negotiation.cc:299] Negotiation successful
0929 15:57:01.729574 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 15:57:01.730221 6762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 15:57:01 dist-test-slave-c51v krb5kdc[6532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697421, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.252@KRBTEST.COM for kudu/127.15.196.252@KRBTEST.COM
Sep 29 15:57:01 dist-test-slave-c51v krb5kdc[6532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.196.253@KRBTEST.COM
I20260929 15:57:01.743647 6766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.730962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39326)
0929 15:57:01.731117 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.731134 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 15:57:01.731225 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:57:01.731630 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:57:01.731634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:57:01.732009 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:57:01.732121 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:01.732128 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.733017 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.733024 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:01.733854 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:01.733864 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.734559 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.734564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:01.734832 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.735431 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:57:01.735447 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:57:01.737613 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:57:01.741098 (+ 3485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:01.741102 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:01.741105 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:01.741525 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:01.741896 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:01.741899 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:01.741901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:01.742024 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:01.742830 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:57:01.742835 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:57:01.742973 (+ 138us) client_negotiation.cc:769] Sending connection context
0929 15:57:01.743094 (+ 121us) client_negotiation.cc:240] Negotiation successful
0929 15:57:01.743290 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":22}
I20260929 15:57:01.744048 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.731174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39326 (local address 127.15.196.252:35449)
0929 15:57:01.731363 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:01.731368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:01.731385 (+ 17us) server_negotiation.cc:407] Connection header received
0929 15:57:01.731442 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:01.731447 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:01.731515 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:01.731614 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:01.732242 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.732906 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.733993 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.734413 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.734550 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.737819 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:57:01.737827 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:57:01.737834 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:57:01.737874 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:57:01.740969 (+ 3095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.741644 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.741651 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.741654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.741790 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.742124 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.742129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.742133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.742667 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:57:01.742811 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:01.743213 (+ 402us) server_negotiation.cc:299] Negotiation successful
0929 15:57:01.743470 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":48}
I20260929 15:57:01.746809 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.731725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42480 (local address 127.15.196.253:36989)
0929 15:57:01.731869 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:01.731874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:01.732005 (+ 131us) server_negotiation.cc:407] Connection header received
0929 15:57:01.732170 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:01.732175 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:01.732233 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:01.732324 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:01.733432 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.734108 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.735464 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.735869 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.736014 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.740755 (+ 4741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:57:01.740763 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:57:01.740768 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:57:01.740807 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:57:01.744148 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.744874 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.744878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.744882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.745033 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.745411 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.745423 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.745427 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.745766 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:57:01.746041 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:01.746229 (+ 188us) server_negotiation.cc:299] Negotiation successful
0929 15:57:01.746475 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 15:57:01.749867 6762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31: Bootstrap starting.
I20260929 15:57:01.751688 16147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 15:57:01.753800 6762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31: Neither blocks nor log segments found. Creating new log.
I20260929 15:57:01.755017 6762 log.cc:824] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31: Log is configured to *not* fsync() on all Append() calls
I20260929 15:57:01.754709 6588 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0705c9f8915749f78b913da2ab21ad6b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50170
I20260929 15:57:01.757717 6762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31: No bootstrap required, opened a new log
I20260929 15:57:01.764606 6762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } }
I20260929 15:57:01.764969 6762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 711d2b55eb2b4d829bfb678218cd7f31, State: Initialized, Role: FOLLOWER
I20260929 15:57:01.765413 6762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } }
I20260929 15:57:01.766377 6768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } } }
I20260929 15:57:01.766744 6768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:57:01.767539 6762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31 [sys.catalog]: configured and running, proceeding with master startup.
W20260929 15:57:01.776517 6779 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 15:57:01.776645 6779 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 15:57:01.774544 6694 master_runner.cc:186] Error getting master registration for 127.15.196.254:38211: OK, instance_id { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" instance_seqno: 1790697420668930 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 15:57:01.785629 6694 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 15:57:01.811823 6613 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } attempt: 4
Sep 29 15:57:01 dist-test-slave-c51v krb5kdc[6532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697420, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.254@KRBTEST.COM for kudu/127.15.196.252@KRBTEST.COM
I20260929 15:57:01.828012 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.812758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39334 (local address 127.15.196.252:35449)
0929 15:57:01.813210 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:01.813214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:01.813229 (+ 15us) server_negotiation.cc:407] Connection header received
0929 15:57:01.813281 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:01.813285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:01.813345 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:01.813423 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:01.814460 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.815643 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.817480 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.818196 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.818318 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.822111 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:57:01.822117 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:57:01.822122 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:57:01.822161 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:57:01.825021 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.825687 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.825696 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.825699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.825815 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.826499 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.826505 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.826509 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.826856 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:57:01.827040 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:01.827418 (+ 378us) server_negotiation.cc:299] Negotiation successful
0929 15:57:01.827625 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314}
I20260929 15:57:01.827997 6616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.812635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39334)
0929 15:57:01.812857 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.812878 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 15:57:01.812998 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:57:01.813510 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:57:01.813516 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:57:01.814058 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:57:01.814287 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:01.814301 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.815798 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.815807 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:01.817339 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:01.817354 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.818357 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.818364 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:01.818583 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.819201 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:57:01.819225 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:57:01.821929 (+ 2704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:57:01.825158 (+ 3229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:01.825163 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:01.825168 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:01.825563 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:01.826246 (+ 683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:01.826250 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:01.826252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:01.826382 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:01.826989 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:57:01.826993 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:57:01.827157 (+ 164us) client_negotiation.cc:769] Sending connection context
0929 15:57:01.827321 (+ 164us) client_negotiation.cc:240] Negotiation successful
0929 15:57:01.827620 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 15:57:01.834327 6613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b: Bootstrap starting.
I20260929 15:57:01.838428 6613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b: Neither blocks nor log segments found. Creating new log.
I20260929 15:57:01.839605 6613 log.cc:824] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b: Log is configured to *not* fsync() on all Append() calls
I20260929 15:57:01.842886 6613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b: No bootstrap required, opened a new log
I20260929 15:57:01.850577 6613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } }
I20260929 15:57:01.851013 6613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0705c9f8915749f78b913da2ab21ad6b, State: Initialized, Role: FOLLOWER
I20260929 15:57:01.851518 6613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } }
I20260929 15:57:01.852505 6782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } } }
I20260929 15:57:01.852895 6782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:57:01.853670 6613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:57:01.853816 6588 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 15:57:01.854038 6588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:57:01.857661 6588 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } }
I20260929 15:57:01.859557 6588 leader_election.cc:288] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [CANDIDATE]: Term 1 election: Requested vote from peers aaaa24c395344612a07fd9593cae4d56 (127.15.196.253:36989), 711d2b55eb2b4d829bfb678218cd7f31 (127.15.196.252:35449)
I20260929 15:57:01.860671 6737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0705c9f8915749f78b913da2ab21ad6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "711d2b55eb2b4d829bfb678218cd7f31"
I20260929 15:57:01.861177 6737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:57:01.865890 6737 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0705c9f8915749f78b913da2ab21ad6b in term 1.
I20260929 15:57:01.866722 6563 leader_election.cc:302] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0705c9f8915749f78b913da2ab21ad6b, 711d2b55eb2b4d829bfb678218cd7f31; no voters:
I20260929 15:57:01.867842 6782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 15:57:01.884649 6782 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [term 1 LEADER]: Becoming Leader. State: Replica: 0705c9f8915749f78b913da2ab21ad6b, State: Running, Role: LEADER
I20260929 15:57:01.885483 6782 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } }
I20260929 15:57:01.889802 6792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0705c9f8915749f78b913da2ab21ad6b. Latest consensus state: current_term: 1 leader_uuid: "0705c9f8915749f78b913da2ab21ad6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } } }
I20260929 15:57:01.890328 6792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [sys.catalog]: This master's current role is: LEADER
I20260929 15:57:01.898382 6800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 15:57:01.902325 6687 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } attempt: 5
I20260929 15:57:01.904736 6800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260929 15:57:01.908316 6567 tablet.cc:2369] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 15:57:01.915807 6716 tablet.cc:2369] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 15:57:01.923539 6737 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31 [term 1 FOLLOWER]: Refusing update from remote peer 0705c9f8915749f78b913da2ab21ad6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:57:01.924418 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.873967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39346 (local address 127.15.196.252:35449)
0929 15:57:01.874197 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:01.874205 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:01.886045 (+ 11840us) server_negotiation.cc:407] Connection header received
0929 15:57:01.886404 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:01.886411 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:01.886501 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:01.886574 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:01.897827 (+ 11253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.898997 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.901546 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.902229 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.902341 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.909159 (+ 6818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:57:01.909165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:57:01.909170 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:57:01.909215 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:57:01.913729 (+ 4514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.917675 (+ 3946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.917680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.917684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.917850 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.920127 (+ 2277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.920133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.920136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.920469 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:57:01.920732 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:01.923819 (+ 3087us) server_negotiation.cc:299] Negotiation successful
0929 15:57:01.924063 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260929 15:57:01.924942 6792 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:57:01.921384 6806 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 15:57:01.934633 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.870711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42488 (local address 127.15.196.253:36989)
0929 15:57:01.871148 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:01.871157 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:01.885360 (+ 14203us) server_negotiation.cc:407] Connection header received
0929 15:57:01.886237 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:01.886243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:01.886347 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:01.886423 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:01.898551 (+ 12128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.899620 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.915837 (+ 16217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.916490 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.916590 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.926371 (+ 9781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:57:01.926377 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:57:01.926382 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:57:01.926424 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:57:01.930174 (+ 3750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.932056 (+ 1882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.932064 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.932071 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.932241 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.932701 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.932709 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.932713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.933082 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:57:01.933323 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:01.933822 (+ 499us) server_negotiation.cc:299] Negotiation successful
0929 15:57:01.934060 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285}
Sep 29 15:57:01 dist-test-slave-c51v krb5kdc[6532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697421, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.196.253@KRBTEST.COM for kudu/127.15.196.252@KRBTEST.COM
I20260929 15:57:01.937649 6617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.870470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50176 (local address 127.15.196.254:38211)
0929 15:57:01.870959 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:01.870968 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:01.879911 (+ 8943us) server_negotiation.cc:407] Connection header received
0929 15:57:01.881084 (+ 1173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:01.881094 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:01.881208 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:01.881301 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:01.890385 (+ 9084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.891903 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.894493 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.895328 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.895462 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.925208 (+ 29746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:57:01.925215 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:57:01.925220 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:57:01.925262 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:57:01.929534 (+ 4272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.930370 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.930377 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.930382 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.930559 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.931010 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.931015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.931017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.931430 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:57:01.931692 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:01.932235 (+ 543us) server_negotiation.cc:299] Negotiation successful
0929 15:57:01.932762 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":309}
I20260929 15:57:01.948364 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.902935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.196.252:35449 (local address 127.0.0.1:39360)
0929 15:57:01.903366 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0929 15:57:01.903386 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 15:57:01.903494 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 15:57:01.911894 (+ 8400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 15:57:01.911901 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 15:57:01.912433 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0929 15:57:01.912689 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:01.912702 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.928557 (+ 15855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.928567 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:01.930127 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 15:57:01.930146 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.931898 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.931908 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 15:57:01.932210 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.932867 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 15:57:01.932900 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0929 15:57:01.937375 (+ 4475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 15:57:01.943899 (+ 6524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:01.943908 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:01.943914 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:01.944563 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:01.945035 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 15:57:01.945041 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 15:57:01.945045 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 15:57:01.945359 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 15:57:01.946596 (+ 1237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 15:57:01.946602 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 15:57:01.946881 (+ 279us) client_negotiation.cc:769] Sending connection context
0929 15:57:01.947142 (+ 261us) client_negotiation.cc:240] Negotiation successful
0929 15:57:01.947678 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278}
I20260929 15:57:01.950469 6804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:01.903720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39360 (local address 127.15.196.252:35449)
0929 15:57:01.911496 (+ 7776us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:01.911501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:01.911529 (+ 28us) server_negotiation.cc:407] Connection header received
0929 15:57:01.911587 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:01.911592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:01.911679 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:01.912848 (+ 1169us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:01.913133 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.914426 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.931020 (+ 16594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:01.931710 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:01.931843 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:01.937621 (+ 5778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 15:57:01.937629 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 15:57:01.937635 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 15:57:01.937672 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 15:57:01.943695 (+ 6023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.944714 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.944719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.944722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.944870 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 15:57:01.945583 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 15:57:01.945590 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 15:57:01.945630 (+ 40us) server_negotiation.cc:967] calling sasl_server_step()
0929 15:57:01.946337 (+ 707us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 15:57:01.948902 (+ 2565us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:01.949226 (+ 324us) server_negotiation.cc:299] Negotiation successful
0929 15:57:01.949655 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7602,"thread_start_us":6497,"threads_started":1}
I20260929 15:57:01.925261 6545 master_runner.cc:186] Error getting master registration for 127.15.196.253:36989: OK, instance_id { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" instance_seqno: 1790697421165432 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 15:57:01.956496 6687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56: Bootstrap starting.
I20260929 15:57:01.959491 6545 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 15:57:01.960959 6687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56: Neither blocks nor log segments found. Creating new log.
I20260929 15:57:01.962479 6687 log.cc:824] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56: Log is configured to *not* fsync() on all Append() calls
I20260929 15:57:01.979522 6768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0705c9f8915749f78b913da2ab21ad6b. Latest consensus state: current_term: 1 leader_uuid: "0705c9f8915749f78b913da2ab21ad6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } } }
I20260929 15:57:01.979838 6768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:57:01.981002 6687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56: No bootstrap required, opened a new log
I20260929 15:57:01.982120 6805 mvcc.cc:205] Tried to move back new op lower bound from 7334696640182583296 to 7334696640050098176. Current Snapshot: MvccSnapshot[applied={T|T < 7334696640182583296}]
I20260929 15:57:02.000669 6687 raft_consensus.cc:348] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } }
I20260929 15:57:02.001464 6687 raft_consensus.cc:729] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaaa24c395344612a07fd9593cae4d56, State: Initialized, Role: FOLLOWER
I20260929 15:57:02.003703 6687 consensus_queue.cc:260] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } }
I20260929 15:57:02.010196 6782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0705c9f8915749f78b913da2ab21ad6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } } }
I20260929 15:57:02.010546 6782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [sys.catalog]: This master's current role is: LEADER
I20260929 15:57:02.016501 6800 catalog_manager.cc:1347] Generated new cluster ID: 8a395fcb1ee7415bbe67ecbc8b3ae38d
I20260929 15:57:02.016631 6800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 15:57:02.018149 6768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0705c9f8915749f78b913da2ab21ad6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } } }
I20260929 15:57:02.018400 6768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:57:02.019824 6812 sys_catalog.cc:455] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } } }
I20260929 15:57:02.020354 6812 sys_catalog.cc:458] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:57:02.021513 6687 sys_catalog.cc:564] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 15:57:02.021632 6662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0705c9f8915749f78b913da2ab21ad6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "aaaa24c395344612a07fd9593cae4d56"
I20260929 15:57:02.022125 6662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56 [term 0 FOLLOWER]: Advancing to term 1
I20260929 15:57:02.025744 6662 raft_consensus.cc:2454] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0705c9f8915749f78b913da2ab21ad6b in term 1.
I20260929 15:57:02.028510 6661 raft_consensus.cc:1261] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56 [term 1 FOLLOWER]: Refusing update from remote peer 0705c9f8915749f78b913da2ab21ad6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 15:57:02.030699 6792 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 15:57:02.053037 6800 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 15:57:02.055505 6800 catalog_manager.cc:1504] Loading token signing keys...
I20260929 15:57:02.076840 6800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b: Generated new TSK 0
I20260929 15:57:02.078321 6800 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 15:57:02.093395 6812 sys_catalog.cc:455] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0705c9f8915749f78b913da2ab21ad6b. Latest consensus state: current_term: 1 leader_uuid: "0705c9f8915749f78b913da2ab21ad6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } } }
I20260929 15:57:02.093341 6820 sys_catalog.cc:455] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0705c9f8915749f78b913da2ab21ad6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } } }
I20260929 15:57:02.093758 6812 sys_catalog.cc:458] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:57:02.093839 6820 sys_catalog.cc:458] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 15:57:02.095237 6792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0705c9f8915749f78b913da2ab21ad6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0705c9f8915749f78b913da2ab21ad6b" member_type: VOTER last_known_addr { host: "127.15.196.254" port: 38211 } } peers { permanent_uuid: "aaaa24c395344612a07fd9593cae4d56" member_type: VOTER last_known_addr { host: "127.15.196.253" port: 36989 } } peers { permanent_uuid: "711d2b55eb2b4d829bfb678218cd7f31" member_type: VOTER last_known_addr { host: "127.15.196.252" port: 35449 } } }
I20260929 15:57:02.095523 6792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0705c9f8915749f78b913da2ab21ad6b [sys.catalog]: This master's current role is: LEADER
I20260929 15:57:02.129765 6829 catalog_manager.cc:1259] Loaded cluster ID: 8a395fcb1ee7415bbe67ecbc8b3ae38d
I20260929 15:57:02.129964 6829 catalog_manager.cc:1552] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56: loading cluster ID for follower catalog manager: success
I20260929 15:57:02.133705 6829 catalog_manager.cc:1574] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56: acquiring CA information for follower catalog manager: success
I20260929 15:57:02.144148 6829 catalog_manager.cc:1602] T 00000000000000000000000000000000 P aaaa24c395344612a07fd9593cae4d56: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 15:57:02.225950 6617 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:57:02.207264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50182 (local address 127.15.196.254:38211)
0929 15:57:02.207534 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.207541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.223935 (+ 16394us) server_negotiation.cc:407] Connection header received
0929 15:57:02.224247 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.224254 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.224345 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.224420 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:02.225687 (+ 1267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50182: BlockingRecv error: recv got EOF from 127.0.0.1:50182 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260929 15:57:02.232553 6804 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:57:02.213469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39368 (local address 127.15.196.252:35449)
0929 15:57:02.213672 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.213678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.226733 (+ 13055us) server_negotiation.cc:407] Connection header received
0929 15:57:02.226916 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.226923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.227035 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.227141 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:02.232272 (+ 5131us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39368: BlockingRecv error: recv got EOF from 127.0.0.1:39368 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260929 15:57:02.234124 6692 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:57:02.218720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42500 (local address 127.15.196.253:36989)
0929 15:57:02.218929 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.218935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.232356 (+ 13421us) server_negotiation.cc:407] Connection header received
0929 15:57:02.232614 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.232620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.232714 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.232785 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:02.233888 (+ 1103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42500: BlockingRecv error: recv got EOF from 127.0.0.1:42500 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 15:57:02.292869 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:02.267790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42506 (local address 127.15.196.253:36989)
0929 15:57:02.268010 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.268017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.284005 (+ 15988us) server_negotiation.cc:407] Connection header received
0929 15:57:02.284271 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.284278 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.284501 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.284703 (+ 202us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:57:02.285474 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.286598 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.289236 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.289967 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.290082 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:02.290888 (+ 806us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:57:02.291708 (+ 820us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:57:02.291876 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:02.292401 (+ 525us) server_negotiation.cc:299] Negotiation successful
0929 15:57:02.292501 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260929 15:57:02.338351 6804 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:57:02.326846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39374 (local address 127.15.196.252:35449)
0929 15:57:02.327048 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.327054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.336116 (+ 9062us) server_negotiation.cc:407] Connection header received
0929 15:57:02.336545 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.336552 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.336713 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.336807 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:02.338041 (+ 1234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39374: BlockingRecv error: recv got EOF from 127.0.0.1:39374 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 15:57:02.390184 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:02.374406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42520 (local address 127.15.196.253:36989)
0929 15:57:02.374620 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.374628 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.382796 (+ 8168us) server_negotiation.cc:407] Connection header received
0929 15:57:02.383411 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.383418 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.383549 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.383629 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:57:02.384391 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.385606 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.387721 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.388418 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.388518 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:02.389177 (+ 659us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:57:02.389370 (+ 193us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:57:02.389469 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:02.389705 (+ 236us) server_negotiation.cc:299] Negotiation successful
0929 15:57:02.389825 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260929 15:57:02.433557 6804 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:57:02.425217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39388 (local address 127.15.196.252:35449)
0929 15:57:02.425432 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.425438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.431938 (+ 6500us) server_negotiation.cc:407] Connection header received
0929 15:57:02.432132 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.432138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.432231 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.432338 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:02.433305 (+ 967us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39388: BlockingRecv error: recv got EOF from 127.0.0.1:39388 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 15:57:02.481745 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:02.466602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42536 (local address 127.15.196.253:36989)
0929 15:57:02.466804 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.466810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.474497 (+ 7687us) server_negotiation.cc:407] Connection header received
0929 15:57:02.475011 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.475018 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.475107 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.475170 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:57:02.475857 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.477003 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.479387 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.480063 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.480159 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:02.480887 (+ 728us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:57:02.481028 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:57:02.481099 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:02.481302 (+ 203us) server_negotiation.cc:299] Negotiation successful
0929 15:57:02.481398 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260929 15:57:02.505645 6804 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:57:02.503371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39390 (local address 127.15.196.252:35449)
0929 15:57:02.503611 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.503620 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.504125 (+ 505us) server_negotiation.cc:407] Connection header received
0929 15:57:02.504354 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.504359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.504449 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.504559 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:02.505388 (+ 829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39390: BlockingRecv error: recv got EOF from 127.0.0.1:39390 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 15:57:02.517998 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:02.512506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42546 (local address 127.15.196.253:36989)
0929 15:57:02.512808 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.512817 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.513316 (+ 499us) server_negotiation.cc:407] Connection header received
0929 15:57:02.513533 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.513539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.513618 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.513714 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:57:02.514123 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.514912 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.516298 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.516772 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.516906 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:02.517284 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:57:02.517386 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:57:02.517483 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:02.517624 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 15:57:02.517704 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
W20260929 15:57:02.531026 6804 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:57:02.529124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39402 (local address 127.15.196.252:35449)
0929 15:57:02.529364 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.529370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.529636 (+ 266us) server_negotiation.cc:407] Connection header received
0929 15:57:02.529901 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.529906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.530000 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.530104 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:02.530848 (+ 744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39402: BlockingRecv error: recv got EOF from 127.0.0.1:39402 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
I20260929 15:57:02.548559 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:02.542974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42556 (local address 127.15.196.253:36989)
0929 15:57:02.543202 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.543212 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.543760 (+ 548us) server_negotiation.cc:407] Connection header received
0929 15:57:02.543981 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.543989 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.544057 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.544169 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:57:02.544560 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.545324 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.546636 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.547163 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.547363 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:02.547756 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:57:02.547894 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:57:02.548027 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:02.548171 (+ 144us) server_negotiation.cc:299] Negotiation successful
0929 15:57:02.548261 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260929 15:57:02.558105 6804 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:57:02.556299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39416 (local address 127.15.196.252:35449)
0929 15:57:02.556513 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.556520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.556730 (+ 210us) server_negotiation.cc:407] Connection header received
0929 15:57:02.556941 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.556954 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.557038 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.557142 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:02.557903 (+ 761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39416: BlockingRecv error: recv got EOF from 127.0.0.1:39416 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 15:57:02.584575 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:02.578731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42566 (local address 127.15.196.253:36989)
0929 15:57:02.578961 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.578973 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.579338 (+ 365us) server_negotiation.cc:407] Connection header received
0929 15:57:02.579539 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.579545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.579639 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.579771 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:57:02.580141 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.581372 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.582626 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.583328 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.583453 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:02.583839 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:57:02.583983 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:57:02.584076 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:02.584198 (+ 122us) server_negotiation.cc:299] Negotiation successful
0929 15:57:02.584281 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260929 15:57:02.592034 6804 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:57:02.589995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39420 (local address 127.15.196.252:35449)
0929 15:57:02.590208 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.590213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.590577 (+ 364us) server_negotiation.cc:407] Connection header received
0929 15:57:02.590755 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.590759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.590833 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.590908 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:02.591782 (+ 874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39420: BlockingRecv error: recv got EOF from 127.0.0.1:39420 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260929 15:57:02.634249 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:02.628095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42576 (local address 127.15.196.253:36989)
0929 15:57:02.628321 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.628328 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.628888 (+ 560us) server_negotiation.cc:407] Connection header received
0929 15:57:02.629163 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.629167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.629270 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.629370 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:57:02.629857 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.630621 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.632551 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.633008 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.633106 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:02.633540 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:57:02.633637 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:57:02.633700 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:02.633847 (+ 147us) server_negotiation.cc:299] Negotiation successful
0929 15:57:02.633911 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260929 15:57:02.642236 6804 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:57:02.640365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39426 (local address 127.15.196.252:35449)
0929 15:57:02.640582 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.640588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.640865 (+ 277us) server_negotiation.cc:407] Connection header received
0929 15:57:02.641046 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.641050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.641132 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.641216 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:02.642028 (+ 812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39426: BlockingRecv error: recv got EOF from 127.0.0.1:39426 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260929 15:57:02.715073 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:02.709941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.15.196.253:36989)
0929 15:57:02.710182 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.710191 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.710546 (+ 355us) server_negotiation.cc:407] Connection header received
0929 15:57:02.710772 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.710776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.710843 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.710948 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:57:02.711394 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.712156 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.713502 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.713888 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.714032 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:02.714398 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:57:02.714497 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:57:02.714589 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:02.714730 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 15:57:02.714805 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260929 15:57:02.724700 6804 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 15:57:02.722859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39436 (local address 127.15.196.252:35449)
0929 15:57:02.723040 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.723048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.723435 (+ 387us) server_negotiation.cc:407] Connection header received
0929 15:57:02.723623 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.723627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.723698 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.723820 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 15:57:02.724541 (+ 721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39436: BlockingRecv error: recv got EOF from 127.0.0.1:39436 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 15:57:02.778674 6779 catalog_manager.cc:1259] Loaded cluster ID: 8a395fcb1ee7415bbe67ecbc8b3ae38d
I20260929 15:57:02.778816 6779 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31: loading cluster ID for follower catalog manager: success
I20260929 15:57:02.781342 6779 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31: acquiring CA information for follower catalog manager: success
I20260929 15:57:02.783140 6779 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 711d2b55eb2b4d829bfb678218cd7f31: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 15:57:02.860948 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:02.856413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42594 (local address 127.15.196.253:36989)
0929 15:57:02.856583 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.856587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.856905 (+ 318us) server_negotiation.cc:407] Connection header received
0929 15:57:02.857078 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.857082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.857150 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.857237 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:57:02.857581 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.858263 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.859504 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.859887 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.859977 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:02.860311 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:57:02.860433 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:57:02.860496 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:02.860612 (+ 116us) server_negotiation.cc:299] Negotiation successful
0929 15:57:02.860671 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 15:57:02.871819 6804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:02.866253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39450 (local address 127.15.196.252:35449)
0929 15:57:02.866406 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.866410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.866722 (+ 312us) server_negotiation.cc:407] Connection header received
0929 15:57:02.866874 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.866877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.867041 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.867189 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:57:02.867557 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.868355 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.869746 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.870215 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.870306 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:02.870763 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:57:02.871332 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:57:02.871451 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:02.871536 (+ 85us) server_negotiation.cc:299] Negotiation successful
0929 15:57:02.871597 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 15:57:02.891912 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:02.878350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42606 (local address 127.15.196.253:36989)
0929 15:57:02.878633 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.878642 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.884834 (+ 6192us) server_negotiation.cc:407] Connection header received
0929 15:57:02.885021 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.885030 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.885148 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.885267 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:57:02.885998 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.887493 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.889286 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.889740 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.889812 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:02.890374 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:57:02.890637 (+ 263us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:57:02.890766 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:02.891532 (+ 766us) server_negotiation.cc:299] Negotiation successful
0929 15:57:02.891617 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260929 15:57:02.894474 6934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:02.877796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50196 (local address 127.15.196.254:38211)
0929 15:57:02.887805 (+ 10009us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.887815 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.887840 (+ 25us) server_negotiation.cc:407] Connection header received
0929 15:57:02.887990 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.887998 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.888265 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.888507 (+ 242us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:57:02.889284 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.890390 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.892283 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.892874 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.893021 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:02.893548 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:57:02.893877 (+ 329us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:57:02.894011 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:02.894105 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 15:57:02.894187 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9836,"thread_start_us":9704,"threads_started":1}
I20260929 15:57:02.896030 6804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 15:57:02.878072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39456 (local address 127.15.196.252:35449)
0929 15:57:02.878292 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 15:57:02.878300 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 15:57:02.889395 (+ 11095us) server_negotiation.cc:407] Connection header received
0929 15:57:02.889683 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 15:57:02.889689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 15:57:02.889804 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 15:57:02.889866 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 15:57:02.890765 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.891617 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.894179 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 15:57:02.894740 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 15:57:02.894877 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 15:57:02.895255 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 15:57:02.895421 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 15:57:02.895498 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 15:57:02.895673 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 15:57:02.895748 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 15:57:02.899559 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 6545
I20260929 15:57:02.900520 6608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:57:03.024626 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 6545
W20260929 15:57:03.045650 6713 connection.cc:535] server connection from 127.0.0.1:39334 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 15:57:03.046072 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 6620
I20260929 15:57:03.047173 6682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:57:03.154619 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 6620
I20260929 15:57:03.177346 16147 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6fGZba/build/asan/bin/kudu pid 6694
I20260929 15:57:03.178400 6757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 15:57:03.286686 16147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6fGZba/build/asan/bin/kudu with pid 6694
2026-09-29T15:57:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4415 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (8887 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (364120 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST